[jira] [Work logged] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?focusedWorklogId=311882=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311882 ] ASF GitHub Bot logged work on HIVE-22188: - Author: ASF GitHub Bot Created on: 13/Sep/19 05:57 Start Date: 13/Sep/19 05:57 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #768: HIVE-22188 Clean up the SemanticAnalyzerFactory URL: https://github.com/apache/hive/pull/768 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311882) Time Spent: 40m (was: 0.5h) > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > Time Spent: 40m > Remaining Estimate: 0h > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > Time Spent: 40m > Remaining Estimate: 0h > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?focusedWorklogId=311881=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311881
]
ASF GitHub Bot logged work on HIVE-22188:
-
Author: ASF GitHub Bot
Created on: 13/Sep/19 05:55
Start Date: 13/Sep/19 05:55
Worklog Time Spent: 10m
Work Description: miklosgergely commented on pull request #768:
HIVE-22188 Clean up the SemanticAnalyzerFactory
URL: https://github.com/apache/hive/pull/768#discussion_r324044354
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
##
@@ -342,33 +342,41 @@ public void analyzeInternal(ASTNode input) throws
SemanticException {
analyzeAlterTableProps(qualified, null, ast, false, false);
} else if (ast.getType() == HiveParser.TOK_ALTERTABLE_DROPPROPERTIES) {
analyzeAlterTableProps(qualified, null, ast, false, true);
- } else if (ast.getType() == HiveParser.TOK_ALTERTABLE_UPDATESTATS) {
+ } else if (ast.getType() == HiveParser.TOK_ALTERTABLE_UPDATESTATS ||
Review comment:
In the long term the whole DDLSemanticAnalyzer will be removed by extracting
each DDL analyzer to it's own class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311881)
Time Spent: 0.5h (was: 20m)
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22199) Ugrade findbugs to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929005#comment-16929005
]
Hive QA commented on HIVE-22199:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense findbugs xml javac javadoc compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18577/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| modules | C: . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18577/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Ugrade findbugs to 3.0.5
>
>
> Key: HIVE-22199
> URL: https://issues.apache.org/jira/browse/HIVE-22199
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22199.01.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22192) Remove ObjectPair classes
[ https://issues.apache.org/jira/browse/HIVE-22192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22192: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Remove ObjectPair classes > - > > Key: HIVE-22192 > URL: https://issues.apache.org/jira/browse/HIVE-22192 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22192.01.patch, HIVE-22192.02.patch > > Time Spent: 20m > Remaining Estimate: 0h > > There are two ObjectPair classes in Hive, which are identical. Actually none > of them is needed, as there is a perfectly fine Pair class with the same > functionalities in the apache common lang jar. So those superfluous classes > should be removed. > Both ObjectPair classes are for internal use only, no other project is > referencing them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22192) Remove ObjectPair classes
[ https://issues.apache.org/jira/browse/HIVE-22192?focusedWorklogId=311878=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311878 ] ASF GitHub Bot logged work on HIVE-22192: - Author: ASF GitHub Bot Created on: 13/Sep/19 05:46 Start Date: 13/Sep/19 05:46 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #770: HIVE-22192 Remove ObjectPair classes (Miklos Gergely reviewed by Jesus Camacho Rodriguez) URL: https://github.com/apache/hive/pull/770 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311878) Time Spent: 20m (was: 10m) > Remove ObjectPair classes > - > > Key: HIVE-22192 > URL: https://issues.apache.org/jira/browse/HIVE-22192 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22192.01.patch, HIVE-22192.02.patch > > Time Spent: 20m > Remaining Estimate: 0h > > There are two ObjectPair classes in Hive, which are identical. Actually none > of them is needed, as there is a perfectly fine Pair class with the same > functionalities in the apache common lang jar. So those superfluous classes > should be removed. > Both ObjectPair classes are for internal use only, no other project is > referencing them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928997#comment-16928997
]
Hive QA commented on HIVE-22188:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980219/HIVE-22188.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16752 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18576/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18576/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18576/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980219 - PreCommit-HIVE-Build
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.4.patch
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, HIVE-22163.4.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928987#comment-16928987
]
Hive QA commented on HIVE-22188:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
30s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} ql: The patch generated 0 new + 701 unchanged - 59
fixed = 701 total (was 760) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
48s{color} | {color:red} ql generated 1 new + 2227 unchanged - 17 fixed = 2228
total (was 2244) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser$DFA236 be a _static_
inner class? At HiveParser.java:inner class? At HiveParser.java:[lines
48714-48727] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18576/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18576/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18576/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928985#comment-16928985
]
Hive QA commented on HIVE-22195:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980209/HIVE-22195.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16755 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18575/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18575/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18575/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980209 - PreCommit-HIVE-Build
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
> URL: https://issues.apache.org/jira/browse/HIVE-22195
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch
>
>
> This could be useful in case when cluster is kerberized, but Zookeeper is not.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928980#comment-16928980
]
Hive QA commented on HIVE-22195:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 61 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
19s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 19s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
9s{color} | {color:red} llap-client: The patch generated 5 new + 49 unchanged -
1 fixed = 54 total (was 50) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} service: The patch generated 1 new + 44 unchanged - 0
fixed = 45 total (was 44) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
20s{color} | {color:red} service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18575/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18575/yetus/patch-mvninstall-service.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18575/yetus/patch-compile-service.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18575/yetus/patch-compile-service.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18575/yetus/diff-checkstyle-llap-client.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18575/yetus/diff-checkstyle-service.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18575/yetus/patch-findbugs-service.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18575/yetus/patch-asflicense-problems.txt
|
| modules | C: common llap-client service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18575/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Configure authentication type for Zookeeper when different from the
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928977#comment-16928977
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980208/HIVE-21844.09.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18574/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18574/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18574/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12980208/HIVE-21844.09.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980208 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21884.01.patch, HIVE-21884.02.patch,
> HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch, Scheduled
> queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928976#comment-16928976
]
Hive QA commented on HIVE-22163:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980204/HIVE-22163.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16753 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
org.apache.hive.jdbc.miniHS2.TestHs2ConnectionMetricsBinary.testOpenConnectionMetrics
(batchId=287)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18573/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18573/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18573/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980204 - PreCommit-HIVE-Build
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, HIVE-22163.4.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=311845=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311845
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 13/Sep/19 01:58
Start Date: 13/Sep/19 01:58
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r323992223
##
File path: ql/src/java/org/apache/hadoop/hive/ql/udf/UDFFromUnixTime.java
##
@@ -31,12 +30,14 @@
/**
* UDFFromUnixTime.
*
+ * @deprecated Replaced by {@link
org.apache.hadoop.hive.ql.udf.generic.GenericUDFFromUnixTime}.
Review comment:
Can this class be removed altogether now that we have
`GenericUDFFromUnixTime`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311845)
Time Spent: 20m (was: 10m)
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=311846=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311846
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 13/Sep/19 01:58
Start Date: 13/Sep/19 01:58
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r323995778
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToUnixTimeStamp.java
##
@@ -117,6 +121,19 @@ protected void initializeInput(ObjectInspector[]
arguments) throws UDFArgumentEx
+ " takes only string/date/timestamp/timestampwltz types. Got
Type:" + arg1OI
.getPrimitiveCategory().name());
}
+
+if (timeZone == null) {
+ timeZone = SessionState.get().getConf().getLocalTimeZone();
+ formatter.setTimeZone(TimeZone.getTimeZone(timeZone));
+}
+ }
+
+ public void configure(MapredContext context) {
Review comment:
Is this for testing purpose only? Can't find who is calling this beside
tests.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311846)
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=311844=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311844
]
ASF GitHub Bot logged work on HIVE-22170:
-
Author: ASF GitHub Bot
Created on: 13/Sep/19 01:58
Start Date: 13/Sep/19 01:58
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on pull request #764: HIVE-22170
URL: https://github.com/apache/hive/pull/764#discussion_r324011653
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFFromUnixTime.java
##
@@ -0,0 +1,173 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.udf.generic;
+
+import java.text.SimpleDateFormat;
+import java.time.ZoneId;
+import java.util.Date;
+import java.util.TimeZone;
+import org.apache.commons.lang.StringUtils;
+import org.apache.hadoop.hive.common.type.TimestampTZUtil;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.apache.hadoop.hive.ql.exec.Description;
+import org.apache.hadoop.hive.ql.exec.MapredContext;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
+import org.apache.hadoop.hive.ql.exec.UDFArgumentLengthException;
+import org.apache.hadoop.hive.ql.metadata.HiveException;
+import org.apache.hadoop.hive.ql.session.SessionState;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector.Category;
+import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters;
+import
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.Converter;
+import org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.IntObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.LongObjectInspector;
+import
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
+import org.apache.hadoop.io.Text;
+
+/**
+ * GenericUDFFromUnixTime.
+ *
+ */
+@Description(name = "from_unixtime",
+value = "_FUNC_(unix_time, format) - returns unix_time in the specified
format",
+extended = "Example:\n"
++ " > SELECT _FUNC_(0, '-MM-dd HH:mm:ss') FROM src LIMIT 1;\n"
++ " '1970-01-01 00:00:00'")
+public class GenericUDFFromUnixTime extends GenericUDF {
+
+ private transient IntObjectInspector inputIntOI;
+ private transient LongObjectInspector inputLongOI;
+ private transient Converter inputTextConverter;
+ private transient ZoneId timeZone;
+ private transient final Text result = new Text();
+
+ private transient SimpleDateFormat formatter = new
SimpleDateFormat("-MM-dd HH:mm:ss");
+ private transient String lastFormat = null;
+
+
+ @Override
+ public ObjectInspector initialize(ObjectInspector[] arguments) throws
UDFArgumentException {
+if (arguments.length < 1) {
+ throw new UDFArgumentLengthException("The function " +
getName().toUpperCase() +
+ "requires at least one argument");
+}
+if (arguments.length > 2) {
+ throw new UDFArgumentLengthException("Too many arguments for the
function " + getName().toUpperCase());
+}
+for (ObjectInspector argument : arguments) {
+ if (argument.getCategory() != Category.PRIMITIVE) {
+throw new UDFArgumentException(getName().toUpperCase() +
+" only takes primitive types, got " + argument.getTypeName());
+ }
+}
+
+PrimitiveObjectInspector arg0OI = (PrimitiveObjectInspector) arguments[0];
+switch (arg0OI.getPrimitiveCategory()) {
+ case INT:
+inputIntOI = (IntObjectInspector) arguments[0];
+break;
+ case LONG:
+inputLongOI = (LongObjectInspector) arguments[0];
+break;
+ default:
+throw new UDFArgumentException("The function " +
getName().toUpperCase()
++ " takes only int/long types for first argument. Got Type:" +
arg0OI.getPrimitiveCategory().name());
+}
+
+if (arguments.length == 2) {
+ PrimitiveObjectInspector arg1OI = (PrimitiveObjectInspector)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928967#comment-16928967
]
Hive QA commented on HIVE-22163:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
28s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 1 new + 208 unchanged - 0
fixed = 209 total (was 208) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18573/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18573/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18573/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, HIVE-22163.4.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
[jira] [Commented] (HIVE-20033) Backport HIVE-19432 to branch-2, branch-3
[
https://issues.apache.org/jira/browse/HIVE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928964#comment-16928964
]
Hive QA commented on HIVE-20033:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12935498/HIVE-20033.03-branch-3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 135 failed/errored test(s), 14435 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=275)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCloseableThreadLocal - did not produce a TEST-*.xml file (likely timed out)
(batchId=334)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=275)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=231)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=239)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=230)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=238)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=233)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHmsServerAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestKuduCliDriver - did not
[jira] [Commented] (HIVE-20033) Backport HIVE-19432 to branch-2, branch-3
[
https://issues.apache.org/jira/browse/HIVE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928948#comment-16928948
]
Hive QA commented on HIVE-20033:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 14s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-18572/patches/PreCommit-HIVE-Build-18572.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18572/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Backport HIVE-19432 to branch-2, branch-3
> -
>
> Key: HIVE-20033
> URL: https://issues.apache.org/jira/browse/HIVE-20033
> Project: Hive
> Issue Type: Bug
>Reporter: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20033.02-branch-3.patch,
> HIVE-20033.03-branch-3.patch, HIVE-20033.1.branch-2.patch,
> HIVE-20033.1.branch-3.patch
>
>
> Backport HIVE-19432 to branch-2, branch-3
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-15408) Support auto-casting non-boolean where condition into boolean
[
https://issues.apache.org/jira/browse/HIVE-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928945#comment-16928945
]
Hive QA commented on HIVE-15408:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980197/HIVE-15408.05.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18571/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18571/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18571/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12980197/HIVE-15408.05.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980197 - PreCommit-HIVE-Build
> Support auto-casting non-boolean where condition into boolean
> -
>
> Key: HIVE-15408
> URL: https://issues.apache.org/jira/browse/HIVE-15408
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-15408.01.patch, HIVE-15408.02.patch,
> HIVE-15408.03.patch, HIVE-15408.04.patch, HIVE-15408.05.patch
>
>
> Hive should support
> {code}
> select 1 where 1
> {code}
> currently it throws an execption (at least with mr)...complaining about
> Integer / Boolean casting difficulties.
> Current behaviour is a bit inconsistent; because using tez / vectorization
> this is supported.
> related tickets:
> HIVE-15089
> HIVE-1478
> HIVE-13659
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928944#comment-16928944
]
Hive QA commented on HIVE-22038:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980189/HIVE-22038.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16752 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testQueueingWithThreads
(batchId=319)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18570/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18570/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18570/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980189 - PreCommit-HIVE-Build
> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch,
> HIVE-22038.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?focusedWorklogId=311799=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311799 ] ASF GitHub Bot logged work on HIVE-22194: - Author: ASF GitHub Bot Created on: 12/Sep/19 22:53 Start Date: 12/Sep/19 22:53 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #769: HIVE-22194 Break up DDLSemanticAnalyzer - extract Privilege related analyzers URL: https://github.com/apache/hive/pull/769#discussion_r323981124 ## File path: ql/src/java/org/apache/hadoop/hive/ql/parse/authorization/HiveAuthorizationTaskFactoryImpl.java ## @@ -28,21 +28,21 @@ import org.apache.hadoop.hive.metastore.api.PrincipalType; import org.apache.hadoop.hive.ql.ErrorMsg; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.privilege.CreateRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.DropRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.GrantDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.GrantRoleDesc; import org.apache.hadoop.hive.ql.ddl.privilege.PrincipalDesc; import org.apache.hadoop.hive.ql.ddl.privilege.PrivilegeDesc; import org.apache.hadoop.hive.ql.ddl.privilege.PrivilegeObjectDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.RevokeDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.RevokeRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.SetRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowCurrentRoleDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowGrantDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowPrincipalsDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowRoleGrantDesc; -import org.apache.hadoop.hive.ql.ddl.privilege.ShowRolesDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.grant.GrantDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.revoke.RevokeDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.create.CreateRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.drop.DropRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.grant.GrantRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.revoke.RevokeRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.set.SetRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.show.ShowRolesDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.role.showcurrent.ShowCurrentRoleDesc; +import org.apache.hadoop.hive.ql.ddl.privilege.show.grant.ShowGrantDesc; Review comment: Maybe we can merge them under `show` package (but not too strongly opinionated about it, up to you). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311799) Time Spent: 0.5h (was: 20m) > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928919#comment-16928919
]
Hive QA commented on HIVE-22038:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18570/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18570/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch,
> HIVE-22038.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?focusedWorklogId=311796=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311796 ] ASF GitHub Bot logged work on HIVE-22194: - Author: ASF GitHub Bot Created on: 12/Sep/19 22:47 Start Date: 12/Sep/19 22:47 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #769: HIVE-22194 Break up DDLSemanticAnalyzer - extract Privilege related analyzers URL: https://github.com/apache/hive/pull/769#discussion_r323979610 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/privilege/role/showcurrent/ShowCurrentRoleAnalyzer.java ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hive.ql.ddl.privilege.role.showcurrent; Review comment: I think we may not need new package and we can use `role.show`, what do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311796) Time Spent: 20m (was: 10m) > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch > > Time Spent: 20m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928917#comment-16928917 ] Jesus Camacho Rodriguez commented on HIVE-22179: [~mgergely], can you fill in the fix version? Thanks > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch, HIVE-22179.02.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22184) Extract LockedDriverState from Driver
[
https://issues.apache.org/jira/browse/HIVE-22184?focusedWorklogId=311781=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311781
]
ASF GitHub Bot logged work on HIVE-22184:
-
Author: ASF GitHub Bot
Created on: 12/Sep/19 22:22
Start Date: 12/Sep/19 22:22
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #767: HIVE-22184
Extract LockedDriverState from Driver
URL: https://github.com/apache/hive/pull/767#discussion_r323973362
##
File path: ql/src/java/org/apache/hadoop/hive/ql/LockedDriverState.java
##
@@ -0,0 +1,149 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql;
+
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.locks.ReentrantLock;
+
+/**
+ * Represents the driver's state. Also has mechanism for locking for the time
of state transitions.
+ */
+public class LockedDriverState {
Review comment:
Let's rename it from `LockedDriverState` to `DriverState`? The private enum
may be just `State`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311781)
Time Spent: 20m (was: 10m)
> Extract LockedDriverState from Driver
> -
>
> Key: HIVE-22184
> URL: https://issues.apache.org/jira/browse/HIVE-22184
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Labels: pull-request-available, refactor-driver
> Attachments: HIVE-22184.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> LockedDriverState is a nested class within Driver, while it is used outside
> of it as well, and it is complex enough to be a class on it's own.
> DriverState should be it's nested class, and transitions / locking should be
> facilitated by functions within it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928908#comment-16928908 ] Jesus Camacho Rodriguez commented on HIVE-22188: Left a small comment. +1 (pending tests) > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > Time Spent: 20m > Remaining Estimate: 0h > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?focusedWorklogId=311780=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311780
]
ASF GitHub Bot logged work on HIVE-22188:
-
Author: ASF GitHub Bot
Created on: 12/Sep/19 22:09
Start Date: 12/Sep/19 22:09
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #768: HIVE-22188
Clean up the SemanticAnalyzerFactory
URL: https://github.com/apache/hive/pull/768#discussion_r323967929
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
##
@@ -342,33 +342,41 @@ public void analyzeInternal(ASTNode input) throws
SemanticException {
analyzeAlterTableProps(qualified, null, ast, false, false);
} else if (ast.getType() == HiveParser.TOK_ALTERTABLE_DROPPROPERTIES) {
analyzeAlterTableProps(qualified, null, ast, false, true);
- } else if (ast.getType() == HiveParser.TOK_ALTERTABLE_UPDATESTATS) {
+ } else if (ast.getType() == HiveParser.TOK_ALTERTABLE_UPDATESTATS ||
Review comment:
We could make this a _switch_ statement too.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311780)
Time Spent: 20m (was: 10m)
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-15408) Support auto-casting non-boolean where condition into boolean
[
https://issues.apache.org/jira/browse/HIVE-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928905#comment-16928905
]
Hive QA commented on HIVE-15408:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980197/HIVE-15408.05.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 35 failed/errored test(s), 16753 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_filter]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_ast] (batchId=96)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fp_literal_arithmetic]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[implicit_cast_during_insert]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_gby4]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_move_tasks_share_dependencies]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_union_src]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_with_join2]
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcs] (batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_deterministic_expr]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_transform]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[structin] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_exists_explain_rewrite]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_in_explain_rewrite]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_multiinsert]
(batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_empty_where]
(batchId=26)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage3]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_transform]
(batchId=148)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multiinsert]
(batchId=151)
org.apache.hadoop.hive.ql.parse.TestIUD.testDeleteWithWhere (batchId=327)
org.apache.hadoop.hive.ql.parse.TestIUD.testStandardInsertIntoTable
(batchId=327)
org.apache.hadoop.hive.ql.parse.TestIUD.testUpdateWithWhereMultiSet
(batchId=327)
org.apache.hadoop.hive.ql.parse.TestIUD.testUpdateWithWhereSingleSet
(batchId=327)
org.apache.hadoop.hive.ql.parse.TestIUD.testUpdateWithWhereSingleSetExpr
(batchId=327)
org.apache.hadoop.hive.ql.parse.TestParseDriver.testParse (batchId=325)
org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testExtractConjuncts
(batchId=327)
org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testExtractSubQueries
(batchId=327)
org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testRewriteOuterQueryWhere
(batchId=327)
org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testRewriteOuterQueryWhere2
(batchId=327)
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthUDFBlacklist.testBlackListedUdfUsage
(batchId=286)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18569/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18569/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18569/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 35 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980197 - PreCommit-HIVE-Build
> Support auto-casting non-boolean where condition into boolean
> -
>
> Key: HIVE-15408
> URL: https://issues.apache.org/jira/browse/HIVE-15408
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-15408.01.patch, HIVE-15408.02.patch,
> HIVE-15408.03.patch, HIVE-15408.04.patch, HIVE-15408.05.patch
>
>
> Hive should support
[jira] [Commented] (HIVE-22192) Remove ObjectPair classes
[ https://issues.apache.org/jira/browse/HIVE-22192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928903#comment-16928903 ] Jesus Camacho Rodriguez commented on HIVE-22192: +1 > Remove ObjectPair classes > - > > Key: HIVE-22192 > URL: https://issues.apache.org/jira/browse/HIVE-22192 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22192.01.patch, HIVE-22192.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > There are two ObjectPair classes in Hive, which are identical. Actually none > of them is needed, as there is a perfectly fine Pair class with the same > functionalities in the apache common lang jar. So those superfluous classes > should be removed. > Both ObjectPair classes are for internal use only, no other project is > referencing them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928898#comment-16928898
]
Hive QA commented on HIVE-21884:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
26s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 61 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
1s{color} | {color:blue} standalone-metastore/metastore-server in master has
180 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
46s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
0s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
38s{color} | {color:red} util in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
20s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
44s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
24s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
25s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
50s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
38s{color} | {color:red} util in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 20s{color}
| {color:red} metastore-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 44s{color}
| {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 24s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 25s{color}
| {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 50s{color}
| {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 38s{color}
| {color:red} util in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
59s{color} | {color:red} standalone-metastore: The patch generated 19 new +
1745 unchanged - 15 fixed = 1764 total (was 1760) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 1 new + 495 unchanged - 0 fixed = 496 total (was
[jira] [Commented] (HIVE-15408) Support auto-casting non-boolean where condition into boolean
[
https://issues.apache.org/jira/browse/HIVE-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928893#comment-16928893
]
Hive QA commented on HIVE-15408:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
16s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18569/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18569/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support auto-casting non-boolean where condition into boolean
> -
>
> Key: HIVE-15408
> URL: https://issues.apache.org/jira/browse/HIVE-15408
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-15408.01.patch, HIVE-15408.02.patch,
> HIVE-15408.03.patch, HIVE-15408.04.patch, HIVE-15408.05.patch
>
>
> Hive should support
> {code}
> select 1 where 1
> {code}
> currently it throws an execption (at least with mr)...complaining about
> Integer / Boolean casting difficulties.
> Current behaviour is a bit inconsistent; because using tez / vectorization
> this is supported.
> related tickets:
> HIVE-15089
> HIVE-1478
> HIVE-13659
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22199: -- Attachment: HIVE-22199.01.patch > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22199.01.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22199: -- Status: Patch Available (was: Open) > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22199.01.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928843#comment-16928843
]
Hive QA commented on HIVE-22059:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980181/HIVE-22059.06.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18567/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18567/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18567/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12980181/HIVE-22059.06.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980181 - PreCommit-HIVE-Build
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch, HIVE-22059.04.patch,
> HIVE-22059.05.patch, HIVE-22059.06.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.databind.ObjectMapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at
[jira] [Commented] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928842#comment-16928842
]
Hive QA commented on HIVE-22059:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980181/HIVE-22059.06.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16752 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18566/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18566/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18566/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980181 - PreCommit-HIVE-Build
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch, HIVE-22059.04.patch,
> HIVE-22059.05.patch, HIVE-22059.06.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.databind.ObjectMapper
> at
[jira] [Commented] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928813#comment-16928813
]
Hive QA commented on HIVE-22059:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18566/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18566/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch, HIVE-22059.04.patch,
> HIVE-22059.05.patch, HIVE-22059.06.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
>
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928801#comment-16928801
]
Hive QA commented on HIVE-21508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980178/HIVE-21508.4.branch-3.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 127 failed/errored test(s), 14412 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=273)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=226)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestCloseableThreadLocal - did not produce a TEST-*.xml file (likely timed out)
(batchId=332)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestDatabaseName - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=273)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=229)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=237)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=228)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=232)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=236)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=228)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=236)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=231)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=232)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHmsServerAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=232)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestJSONMessageDeserializer - did not
[jira] [Commented] (HIVE-22197) Common Merge join throwing class cast exception
[ https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928790#comment-16928790 ] Vineet Garg commented on HIVE-22197: [~maheshk114] Thanks for providing the patch for this. Would you mind adding a test case? > Common Merge join throwing class cast exception > > > Key: HIVE-22197 > URL: https://issues.apache.org/jira/browse/HIVE-22197 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22197.01.patch > > > In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed > and stored in the writable format, but the object inspector is initialized to > default. So when join operator is fetching the data from dummy store > operator, its getting the OI is Long and the row as LongWritable. This is > causing the class cast exception. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928731#comment-16928731
]
Hive QA commented on HIVE-21508:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 17s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-18565/patches/PreCommit-HIVE-Build-18565.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18565/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Attachment: (was: HIVE-22188.01.patch) > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Attachment: HIVE-22188.01.patch > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928727#comment-16928727
]
Hive QA commented on HIVE-22188:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980175/HIVE-22188.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16752 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=111)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18564/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18564/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18564/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980175 - PreCommit-HIVE-Build
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Attachment: HIVE-20983.3.patch > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, > HIVE-20983.3.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Status: In Progress (was: Patch Available) > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22199: -- Issue Type: Improvement (was: Bug) > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[ https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22195: -- Attachment: (was: HIVE-22195.3.patch) > Configure authentication type for Zookeeper when different from the default > cluster wide > > > Key: HIVE-22195 > URL: https://issues.apache.org/jira/browse/HIVE-22195 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch > > > This could be useful in case when cluster is kerberized, but Zookeeper is not. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[ https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22195: -- Attachment: HIVE-22195.3.patch > Configure authentication type for Zookeeper when different from the default > cluster wide > > > Key: HIVE-22195 > URL: https://issues.apache.org/jira/browse/HIVE-22195 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch, > HIVE-22195.3.patch > > > This could be useful in case when cluster is kerberized, but Zookeeper is not. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928689#comment-16928689
]
Hive QA commented on HIVE-22188:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} ql: The patch generated 0 new + 701 unchanged - 59
fixed = 701 total (was 760) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
57s{color} | {color:red} ql generated 1 new + 2227 unchanged - 17 fixed = 2228
total (was 2244) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser$DFA236 be a _static_
inner class? At HiveParser.java:inner class? At HiveParser.java:[lines
48714-48727] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18564/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18564/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18564/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-20683) Add the Ability to push Dynamic Between and Bloom filters to Druid
[
https://issues.apache.org/jira/browse/HIVE-20683?focusedWorklogId=311489=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311489
]
ASF GitHub Bot logged work on HIVE-20683:
-
Author: ASF GitHub Bot
Created on: 12/Sep/19 16:04
Start Date: 12/Sep/19 16:04
Worklog Time Spent: 10m
Work Description: b-slim commented on issue #723: [HIVE-20683] Add the
Ability to push Dynamic Between and Bloom filters to Druid
URL: https://github.com/apache/hive/pull/723#issuecomment-530894335
LGTM thanks
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311489)
Time Spent: 5h 20m (was: 5h 10m)
> Add the Ability to push Dynamic Between and Bloom filters to Druid
> --
>
> Key: HIVE-20683
> URL: https://issues.apache.org/jira/browse/HIVE-20683
> Project: Hive
> Issue Type: New Feature
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20683.1.patch, HIVE-20683.10.patch,
> HIVE-20683.2.patch, HIVE-20683.3.patch, HIVE-20683.4.patch,
> HIVE-20683.5.patch, HIVE-20683.6.patch, HIVE-20683.8.patch, HIVE-20683.patch
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> For optimizing joins, Hive generates BETWEEN filter with min-max and BLOOM
> filter for filtering one side of semi-join.
> Druid 0.13.0 will have support for Bloom filters (Added via
> https://github.com/apache/incubator-druid/pull/6222)
> Implementation details -
> # Hive generates and passes the filters as part of 'filterExpr' in TableScan.
> # DruidQueryBasedRecordReader gets this filter passed as part of the conf.
> # During execution phase, before sending the query to druid in
> DruidQueryBasedRecordReader we will deserialize this filter, translate it
> into a DruidDimFilter and add it to existing DruidQuery. Tez executor
> already ensures that when we start reading results from the record reader,
> all the dynamic values are initialized.
> # Explaining a druid query also prints the query sent to druid as
> {{druid.json.query}}. We also need to make sure to update the druid query
> with the filters. During explain we do not have the actual values for the
> dynamic values, so instead of values we will print the dynamic expression
> itself as part of druid query.
> Note:- This work needs druid to be updated to version 0.13.0
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?focusedWorklogId=311487=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311487 ] ASF GitHub Bot logged work on HIVE-20983: - Author: ASF GitHub Bot Created on: 12/Sep/19 16:01 Start Date: 12/Sep/19 16:01 Worklog Time Spent: 10m Work Description: mustafaiman commented on pull request #771: HIVE-20983 URL: https://github.com/apache/hive/pull/771 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311487) Remaining Estimate: 0h Time Spent: 10m > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-20983: -- Labels: pull-request-available (was: ) > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch > > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928667#comment-16928667
]
Hive QA commented on HIVE-21449:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980170/HIVE-21449.2.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16756 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18563/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18563/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18563/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980170 - PreCommit-HIVE-Build
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[ https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22195: -- Attachment: HIVE-22195.2.patch > Configure authentication type for Zookeeper when different from the default > cluster wide > > > Key: HIVE-22195 > URL: https://issues.apache.org/jira/browse/HIVE-22195 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch > > > This could be useful in case when cluster is kerberized, but Zookeeper is not. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21844.09.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21884.01.patch, HIVE-21884.02.patch, > HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22199) Ugrade findbugs to 3.0.5
[ https://issues.apache.org/jira/browse/HIVE-22199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22199: - > Ugrade findbugs to 3.0.5 > > > Key: HIVE-22199 > URL: https://issues.apache.org/jira/browse/HIVE-22199 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22192) Remove ObjectPair classes
[ https://issues.apache.org/jira/browse/HIVE-22192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22192: -- Labels: pull-request-available (was: ) > Remove ObjectPair classes > - > > Key: HIVE-22192 > URL: https://issues.apache.org/jira/browse/HIVE-22192 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22192.01.patch, HIVE-22192.02.patch > > > There are two ObjectPair classes in Hive, which are identical. Actually none > of them is needed, as there is a perfectly fine Pair class with the same > functionalities in the apache common lang jar. So those superfluous classes > should be removed. > Both ObjectPair classes are for internal use only, no other project is > referencing them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22192) Remove ObjectPair classes
[ https://issues.apache.org/jira/browse/HIVE-22192?focusedWorklogId=311453=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311453 ] ASF GitHub Bot logged work on HIVE-22192: - Author: ASF GitHub Bot Created on: 12/Sep/19 14:47 Start Date: 12/Sep/19 14:47 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #770: HIVE-22192 Remove ObjectPair classes URL: https://github.com/apache/hive/pull/770 There are two ObjectPair classes in Hive, which are identical. Actually none of them is needed, as there is a perfectly fine Pair class with the same functionalities in the apache common lang jar. So those superfluous classes should be removed. Both ObjectPair classes are for internal use only, no other project is referencing them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311453) Remaining Estimate: 0h Time Spent: 10m > Remove ObjectPair classes > - > > Key: HIVE-22192 > URL: https://issues.apache.org/jira/browse/HIVE-22192 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22192.01.patch, HIVE-22192.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > There are two ObjectPair classes in Hive, which are identical. Actually none > of them is needed, as there is a perfectly fine Pair class with the same > functionalities in the apache common lang jar. So those superfluous classes > should be removed. > Both ObjectPair classes are for internal use only, no other project is > referencing them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928599#comment-16928599
]
Hive QA commented on HIVE-21449:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 6 new + 11 unchanged - 0 fixed
= 17 total (was 11) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
4s{color} | {color:red} ql generated 27 new + 2217 unchanged - 27 fixed = 2244
total (was 2244) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to KW_IN246 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionAtom(CommonTree)
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceSimilarExpressionAtom(CommonTree)
At HiveParser_IdentifiersParser.java:[line 9707] |
| | Dead store to LPAREN236 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At HiveParser_IdentifiersParser.java:[line 9209] |
| | Dead store to RPAREN238 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.subQueryExpression()
At HiveParser_IdentifiersParser.java:[line 9216] |
| | Redundant nullcheck of sql11ReservedKeywordsUsedAsFunctionName80, which
is known to be non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.functionName()
Redundant null check at HiveParser_IdentifiersParser.java:is known to be
non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.functionName()
Redundant null check at HiveParser_IdentifiersParser.java:[line 3954] |
| | Redundant nullcheck of nonReserved316, which is known to be non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier()
Redundant null check at HiveParser_IdentifiersParser.java:is known to be
non-null in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.identifier()
Redundant null check at HiveParser_IdentifiersParser.java:[line 12456] |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA22
be a _static_ inner class? At HiveParser_IdentifiersParser.java:inner class?
At HiveParser_IdentifiersParser.java:[lines 32334-32347] |
| | Dead store to LA32_100 in
org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA32.specialStateTransition(int,
IntStream) At
HiveParser_IdentifiersParser.java:org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser$DFA32.specialStateTransition(int,
IntStream) At
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.4.patch
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, HIVE-22163.4.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, HIVE-22163.4.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, HIVE-22163.4.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20033) Backport HIVE-19432 to branch-2, branch-3
[ https://issues.apache.org/jira/browse/HIVE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928576#comment-16928576 ] Peter Vary commented on HIVE-20033: --- [~abstractdog]: Do we plan to commit this change? > Backport HIVE-19432 to branch-2, branch-3 > - > > Key: HIVE-20033 > URL: https://issues.apache.org/jira/browse/HIVE-20033 > Project: Hive > Issue Type: Bug >Reporter: Teddy Choi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20033.02-branch-3.patch, > HIVE-20033.03-branch-3.patch, HIVE-20033.1.branch-2.patch, > HIVE-20033.1.branch-3.patch > > > Backport HIVE-19432 to branch-2, branch-3 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928572#comment-16928572
]
Hive QA commented on HIVE-22194:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980166/HIVE-22194.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18562/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18562/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18562/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12980166/HIVE-22194.02.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980166 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Privilege related analyzers
> --
>
> Key: HIVE-22194
> URL: https://issues.apache.org/jira/browse/HIVE-22194
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the privilege related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22192) Remove ObjectPair classes
[
https://issues.apache.org/jira/browse/HIVE-22192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928570#comment-16928570
]
Hive QA commented on HIVE-22192:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980159/HIVE-22192.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16752 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18561/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18561/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18561/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980159 - PreCommit-HIVE-Build
> Remove ObjectPair classes
> -
>
> Key: HIVE-22192
> URL: https://issues.apache.org/jira/browse/HIVE-22192
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22192.01.patch, HIVE-22192.02.patch
>
>
> There are two ObjectPair classes in Hive, which are identical. Actually none
> of them is needed, as there is a perfectly fine Pair class with the same
> functionalities in the apache common lang jar. So those superfluous classes
> should be removed.
> Both ObjectPair classes are for internal use only, no other project is
> referencing them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22165) Synchronisation introduced by HIVE-14296 on SessionManager.closeSession causes high latency in a busy hive server
[ https://issues.apache.org/jira/browse/HIVE-22165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928558#comment-16928558 ] Amruth S commented on HIVE-22165: - It will be great if this can be pulled in. > Synchronisation introduced by HIVE-14296 on SessionManager.closeSession > causes high latency in a busy hive server > - > > Key: HIVE-22165 > URL: https://issues.apache.org/jira/browse/HIVE-22165 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 2.1.0, 2.3.2 >Reporter: Amruth S >Assignee: Amruth S >Priority: Major > Attachments: HIVE-22165.patch, HIVE-22165.patch.1 > > > HIVE-14296 introduces this > [commit|https://github.com/apache/hive/commit/477a47d3b4b9e3da3c22465217c2024588f7f000] > which adds synchronization to SessionManager.closeSession. > And it looks like it is used only for logging purposes. > In a busy hive server where 5-10 sessions are created closed every second, an > increase in latency of any other downstream services (Zk, HDFS) causes a > queueing effect (lot of threads getting blocked on > SessionManager.closeSession) creating an induced latency of 3-5 minutes at > times for just closing the session. > Since the gauge (MetricsConstant.HS2_OPEN_SESSIONS) is already tracking the > open session counts, the synchronization (along with the additional logging) > can be removed without any functionality losses. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22192) Remove ObjectPair classes
[
https://issues.apache.org/jira/browse/HIVE-22192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928542#comment-16928542
]
Hive QA commented on HIVE-22192:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
24s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 61 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} serde in master has 193 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
8s{color} | {color:blue} standalone-metastore/metastore-server in master has
180 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} hcatalog/webhcat/java-client in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} standalone-metastore/metastore-common: The patch
generated 0 new + 410 unchanged - 3 fixed = 410 total (was 413) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} common: The patch generated 0 new + 0 unchanged - 1
fixed = 0 total (was 1) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} serde: The patch generated 0 new + 3 unchanged - 3
fixed = 3 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 662 unchanged - 2 fixed = 663 total (was 664) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 1 new + 1131 unchanged - 24
fixed = 1132 total (was 1155) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch java-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s{color} | {color:green} metastore-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} serde in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
13s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 178 unchanged - 2 fixed = 178 total (was 180) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
[jira] [Updated] (HIVE-22198) set parallel to run an unoin-all SQL and result missing a part of data
[ https://issues.apache.org/jira/browse/HIVE-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuGuangMing updated HIVE-22198: --- Summary: set parallel to run an unoin-all SQL and result missing a part of data (was: set parallel to run an unoin all SQL result loss data) > set parallel to run an unoin-all SQL and result missing a part of data > -- > > Key: HIVE-22198 > URL: https://issues.apache.org/jira/browse/HIVE-22198 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: LuGuangMing >Priority: Major > Attachments: test-parallel-skewJoin-unoinall.sql > > > connect hiveserver to set > hive.auto.convert.join=false,hive.optimize.skewjoin=true, > hive.exec.parallel=true, and run an unoin all with two child join query, > result is wrong, it's missing data. details check attatchment sql file > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-15408) Support auto-casting non-boolean where condition into boolean
[
https://issues.apache.org/jira/browse/HIVE-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-15408:
Attachment: HIVE-15408.05.patch
> Support auto-casting non-boolean where condition into boolean
> -
>
> Key: HIVE-15408
> URL: https://issues.apache.org/jira/browse/HIVE-15408
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-15408.01.patch, HIVE-15408.02.patch,
> HIVE-15408.03.patch, HIVE-15408.04.patch, HIVE-15408.05.patch
>
>
> Hive should support
> {code}
> select 1 where 1
> {code}
> currently it throws an execption (at least with mr)...complaining about
> Integer / Boolean casting difficulties.
> Current behaviour is a bit inconsistent; because using tez / vectorization
> this is supported.
> related tickets:
> HIVE-15089
> HIVE-1478
> HIVE-13659
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22198) set parallel to run an unoin all SQL result loss data
[ https://issues.apache.org/jira/browse/HIVE-22198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LuGuangMing updated HIVE-22198: --- Description: connect hiveserver to set hive.auto.convert.join=false,hive.optimize.skewjoin=true, hive.exec.parallel=true, and run an unoin all with two child join query, result is wrong, it's missing data. details check attatchment sql file was: connect hiveserver to set hive.auto.convert.join=false,hive.optimize.skewjoin=true, hive.exec.parallel=true, and run an unoin all with two child join query, result is wrong, it's missing data; > set parallel to run an unoin all SQL result loss data > - > > Key: HIVE-22198 > URL: https://issues.apache.org/jira/browse/HIVE-22198 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: LuGuangMing >Priority: Major > Attachments: test-parallel-skewJoin-unoinall.sql > > > connect hiveserver to set > hive.auto.convert.join=false,hive.optimize.skewjoin=true, > hive.exec.parallel=true, and run an unoin all with two child join query, > result is wrong, it's missing data. details check attatchment sql file > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22197) Common Merge join throwing class cast exception
[
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928502#comment-16928502
]
Hive QA commented on HIVE-22197:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980153/HIVE-22197.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16752 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18560/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18560/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18560/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980153 - PreCommit-HIVE-Build
> Common Merge join throwing class cast exception
>
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed
> and stored in the writable format, but the object inspector is initialized to
> default. So when join operator is fetching the data from dummy store
> operator, its getting the OI is Long and the row as LongWritable. This is
> causing the class cast exception.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22197) Common Merge join throwing class cast exception
[
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928480#comment-16928480
]
Hive QA commented on HIVE-22197:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} ql: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18560/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18560/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18560/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Common Merge join throwing class cast exception
>
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed
> and stored in the writable format, but the object inspector is initialized to
> default. So when join operator is fetching the data from dummy store
> operator, its getting the OI is Long and the row as LongWritable. This is
> causing the class cast exception.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[ https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22038: Attachment: HIVE-22038.02.patch > Fix memory related sideeffects of opening/closing sessions > -- > > Key: HIVE-22038 > URL: https://issues.apache.org/jira/browse/HIVE-22038 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch, > HIVE-22038.02.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?focusedWorklogId=311359=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311359 ] ASF GitHub Bot logged work on HIVE-22194: - Author: ASF GitHub Bot Created on: 12/Sep/19 11:35 Start Date: 12/Sep/19 11:35 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #769: HIVE-22194 Break up DDLSemanticAnalyzer - extract Privilege related analyzers URL: https://github.com/apache/hive/pull/769 DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: have a separate class for each analyzers have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #2: extract all the privilege related analyzers from DDLSemanticAnalyzer, and move them under the new package. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311359) Remaining Estimate: 0h Time Spent: 10m > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22194: -- Labels: pull-request-available refactor-ddl (was: refactor-ddl) > Break up DDLSemanticAnalyzer - extract Privilege related analyzers > -- > > Key: HIVE-22194 > URL: https://issues.apache.org/jira/browse/HIVE-22194 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch > > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the privilege related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928462#comment-16928462
]
Hive QA commented on HIVE-22079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980137/HIVE-22079.4.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18559/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18559/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18559/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-12 11:29:22.028
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18559/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-12 11:29:22.031
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 101d4d6 HIVE-22126: hive-exec packaging should shade guava
(Laszlo Pinter reviewed by Adam Szita, Peter Vary)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 101d4d6 HIVE-22126: hive-exec packaging should shade guava
(Laszlo Pinter reviewed by Adam Szita, Peter Vary)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-12 11:29:23.101
+ rm -rf ../yetus_PreCommit-HIVE-Build-18559
+ mkdir ../yetus_PreCommit-HIVE-Build-18559
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18559
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18559/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/ql/src/java/org/apache/hadoop/hive/ql/lib/DefaultGraphWalker.java:
does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/lib/ExpressionWalker.java: does
not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/ppd/ExprWalkerProcFactory.java:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out: does not
exist in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc6894725184725796936.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc6894725184725796936.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc1146062031832990439.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version
[jira] [Commented] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928461#comment-16928461
]
Hive QA commented on HIVE-22194:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980166/HIVE-22194.02.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16752 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18558/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18558/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18558/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980166 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Privilege related analyzers
> --
>
> Key: HIVE-22194
> URL: https://issues.apache.org/jira/browse/HIVE-22194
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the privilege related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-15408) Support auto-casting non-boolean where condition into boolean
[
https://issues.apache.org/jira/browse/HIVE-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-15408:
Attachment: HIVE-15408.04.patch
> Support auto-casting non-boolean where condition into boolean
> -
>
> Key: HIVE-15408
> URL: https://issues.apache.org/jira/browse/HIVE-15408
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-15408.01.patch, HIVE-15408.02.patch,
> HIVE-15408.03.patch, HIVE-15408.04.patch
>
>
> Hive should support
> {code}
> select 1 where 1
> {code}
> currently it throws an execption (at least with mr)...complaining about
> Integer / Boolean casting difficulties.
> Current behaviour is a bit inconsistent; because using tez / vectorization
> this is supported.
> related tickets:
> HIVE-15089
> HIVE-1478
> HIVE-13659
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21975) Fix incremental compilation
[
https://issues.apache.org/jira/browse/HIVE-21975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928441#comment-16928441
]
Zoltan Haindrich commented on HIVE-21975:
-
I think the problem areises mostly from these "extends Serializable" things...
my opinion is that "extends Serializable" doesn't add any value/restriction or
what so ever - since Task needs a [type argument which is
Serializable|https://github.com/apache/hive/blob/101d4d66bf0ba56502e0e5689a31cb0ce72ea03e/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java#L55]
{quote}
I just started on the Hive project and hit the same issue. Also hit the issue
in Driver.java.
{quote}
yeah...I usually just delete the ql/target directory when I bump into
thisit would be great to have it fixed...
> Fix incremental compilation
> ---
>
> Key: HIVE-21975
> URL: https://issues.apache.org/jira/browse/HIVE-21975
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> we have an incremental compilation issue around SA ; mostly because of "?
> extends Serializable"
> it could be reproduced with:
> {code}
> git clean -dfx
> mvn install -pl ql -am -DskipTests
> touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
> mvn install -pl ql -DskipTests
> {code}
> error is:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile
> (default-compile) on project hive-exec: Compilation failure: Compilation
> failure:
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[12573,60]
> incompatible types: java.util.List extends java.io.Serializable>> cannot be converted to
> java.util.List>
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[15187,49]
> incompatible types: java.util.List>
> cannot be converted to java.util.List extends java.io.Serializable>>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22194) Break up DDLSemanticAnalyzer - extract Privilege related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928437#comment-16928437
]
Hive QA commented on HIVE-22194:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 289 unchanged - 4
fixed = 289 total (was 293) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18558/dev-support/hive-personality.sh
|
| git revision | master / 101d4d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18558/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLSemanticAnalyzer - extract Privilege related analyzers
> --
>
> Key: HIVE-22194
> URL: https://issues.apache.org/jira/browse/HIVE-22194
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Attachments: HIVE-22194.01.patch, HIVE-22194.02.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the privilege related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22059:
Attachment: (was: HIVE-22059.06.patch)
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch, HIVE-22059.04.patch,
> HIVE-22059.05.patch, HIVE-22059.06.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.databind.ObjectMapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 19 more
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1 (state=08S01,code=2)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22059:
Attachment: HIVE-22059.06.patch
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch, HIVE-22059.04.patch,
> HIVE-22059.05.patch, HIVE-22059.06.patch, HIVE-22059.06.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.databind.ObjectMapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 19 more
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1 (state=08S01,code=2)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21884.10.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, > HIVE-21884.09.patch, HIVE-21884.10.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22059:
Attachment: HIVE-22059.06.patch
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch, HIVE-22059.04.patch,
> HIVE-22059.05.patch, HIVE-22059.06.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.databind.ObjectMapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 19 more
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1 (state=08S01,code=2)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928426#comment-16928426
]
Hive QA commented on HIVE-20113:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980122/HIVE-20113.8.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 16752 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18557/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18557/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18557/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980122 - PreCommit-HIVE-Build
> Shuffle avoidance: Disable 1-1 edges for sorted shuffle
>
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch,
> HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch,
> HIVE-20113.5.patch, HIVE-20113.6.patch, HIVE-20113.7.patch, HIVE-20113.8.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to
> have a final merge at all, it should open a single compressed file and write
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to
> disable the optimization for sorted shuffle edges and stop rewriting the
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: HIVE-21508.4.branch-3.1.patch
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-21508:
--
Fix Version/s: 2.4.0
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928419#comment-16928419
]
Peter Vary commented on HIVE-21508:
---
[~ananamj]: Pushed to branch-2
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22126) hive-exec packaging should shade guava
[ https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-22126: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the patch [~lpinter], and [~szita] for the review! > hive-exec packaging should shade guava > -- > > Key: HIVE-22126 > URL: https://issues.apache.org/jira/browse/HIVE-22126 > Project: Hive > Issue Type: Bug >Reporter: Vihang Karajgaonkar >Assignee: Laszlo Pinter >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22126.01.patch > > > The ql/pom.xml includes complete guava library into hive-exec.jar > https://github.com/apache/hive/blob/master/ql/pom.xml#L990 This causes a > problems for downstream clients of hive which have hive-exec.jar in their > classpath since they are pinned to the same guava version as that of hive. > We should shade guava classes so that other components which depend on > hive-exec can independently use a different version of guava as needed. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928400#comment-16928400
]
Hive QA commented on HIVE-20113:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 1 new + 347 unchanged - 1
fixed = 348 total (was 348) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18557/dev-support/hive-personality.sh
|
| git revision | master / 5b1afb7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18557/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18557/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Shuffle avoidance: Disable 1-1 edges for sorted shuffle
>
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch,
> HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch,
> HIVE-20113.5.patch, HIVE-20113.6.patch, HIVE-20113.7.patch, HIVE-20113.8.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to
> have a final merge at all, it should open a single compressed file and write
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to
> disable the optimization for sorted shuffle edges and stop rewriting the
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Attachment: (was: HIVE-22188.01.patch) > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Attachment: HIVE-22188.01.patch > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928381#comment-16928381
]
Hive QA commented on HIVE-22188:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980113/HIVE-22188.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16752 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid2]
(batchId=163)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18556/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18556/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18556/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980113 - PreCommit-HIVE-Build
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928382#comment-16928382
]
Ana Jalba commented on HIVE-21508:
--
The test results are weird, because locally, all the cli tests pass. I tested
this with Java 8, because Java 7 gives a compilation error for
{{hive-serde.TimestampWritable.java}} about package {{java.time.format}},
{{DateTimeFormatter}} and {{toLocalDateTime()}} not existing.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[ https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928378#comment-16928378 ] Denys Kuzmenko commented on HIVE-22195: --- Thank you, [~pvary]. Just wanted to verify the approach. I will definitely refactor the code and add some tests. > Configure authentication type for Zookeeper when different from the default > cluster wide > > > Key: HIVE-22195 > URL: https://issues.apache.org/jira/browse/HIVE-22195 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22195.1.patch > > > This could be useful in case when cluster is kerberized, but Zookeeper is not. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22177) Beeline query for special character like £ does not give back result
[
https://issues.apache.org/jira/browse/HIVE-22177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928343#comment-16928343
]
philipse commented on HIVE-22177:
-
hi,
it seems the special character is not the standard UTF-8 character, so i try
the following tests.
1) i upload the file with the special character to an hive location(hdfs
location) ,the hive query or presto works fine
2) i use an reflect function to encoder it as the UTF-8, and the hive query
works fine
{code:java}
// INSERT INTO t1 select reflect('java.net.URLEncoder', 'encode','￡' , "UTF-8");
{code}
{code:java}
// SELECT reflect('java.net.URLDecoder', 'decode',test , "UTF-8") FROM t1
WHERE reflect('java.net.URLDecoder', 'decode',test , "UTF-8") ='￡';
+--+--+
| _c0 |
+--+--+
| ￡|
| ￡|
| ￡|
+--+--+
3 rows selected (0.245 seconds)
{code}
the above is my advice,please correct me if i miss sth.
> Beeline query for special character like £ does not give back result
>
>
> Key: HIVE-22177
> URL: https://issues.apache.org/jira/browse/HIVE-22177
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.3.6
> Environment: Cloudera 5.15.1 Hive 1.1.0
> beeline and Impala
>Reporter: Sanjay Mishra
>Priority: Major
>
> I have a table in hive with a column having special character like £. When I
> try to a query as simple as 'SELECT * FROM TABLE WHERE COL = "£"; ', the
> query does not come back with any result.
> The table data is UTF-8 encoded. The sql file that contains the above sql and
> run via beeline is also UTF-8 encoded.
> Interestingly, if I try to create the run the same query using impala against
> the same table, I get back the columns containing the £ symbol.
> Also when the data from the table is exported (SELECT * FROM THE TABLE and
> save the output to csv file) the column with special character is shown as
> NULL.
>
> Steps to replicate:
> # create test.sql file with the below content:
> #
> {code:java}
> CREATE TABLE T1 (test String);
> INSERT INTO T1 VALUES('£');
> {code}
> # beeline>!run test.sql
> # create another file test_select.sql with the below content:
> #
> {code:java}
> SELECT * FROM T1 WHERE test='£';{code}
> # beeline>!run test_select.sql
> # No results will be returned back
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928340#comment-16928340
]
Hive QA commented on HIVE-22188:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2244 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} ql: The patch generated 0 new + 701 unchanged - 59
fixed = 701 total (was 760) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
59s{color} | {color:red} ql generated 1 new + 2227 unchanged - 17 fixed = 2228
total (was 2244) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser$DFA236 be a _static_
inner class? At HiveParser.java:inner class? At HiveParser.java:[lines
48714-48727] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18556/dev-support/hive-personality.sh
|
| git revision | master / 5b1afb7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18556/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18556/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928310#comment-16928310
]
Hive QA commented on HIVE-20983:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980114/HIVE-20983.2.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16752 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18555/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18555/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18555/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980114 - PreCommit-HIVE-Build
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch
>
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
