[jira] [Commented] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog
[ https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932072#comment-16932072 ] Ashutosh Chauhan commented on HIVE-22213: - +1 > TxnHander cleanupRecords should only clean records belonging to default > catalog > --- > > Key: HIVE-22213 > URL: https://issues.apache.org/jira/browse/HIVE-22213 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22213.1.patch > > > Currently it removes record for given database and given table without > checking for the catalog, as a result it can end up removing records when it > shouldn't. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932068#comment-16932068
]
Hive QA commented on HIVE-22191:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980517/HIVE-22191.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 180 failed/errored test(s), 16760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ptf_general_queries]
(batchId=300)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ptf_orcfile]
(batchId=300)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ptf_parquetfile]
(batchId=300)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ptf_persistence]
(batchId=300)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ptf_rcfile]
(batchId=300)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ptf_seqfile]
(batchId=300)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_windowing]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer12]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_colname]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_1] (batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_groupby]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_windowing]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_windowing_no_cbo]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_window]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[leadlag] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[leadlag_queries]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[localtimezone]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[outer_reference_windowed]
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcs] (batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_windowing1]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_windowing2]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptf_decimal] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptf_general_queries]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptf_rcfile] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptf_seqfile] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptfgroupbyjoin]
(batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_basic]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin2] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin5] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_in_explain_rewrite]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_unqualcolumnrefs]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[tablevalues] (batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_windowing_expressions]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[topn] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_binarysetfunctions]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_binarysetfunctions_no_cbo]
(batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union36] (batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_6_subq]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[unionall_lateralview]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_adjust_rowcontainer_sz]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_columnPruning]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_decimal]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_distinct]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_duplicate]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_expressions]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_gby2]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_multipartitioning]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_navfn]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_ntile]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_order_null]
(batchId=60)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Fix Version/s: (was: 4.0.0)
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932065#comment-16932065
]
Himanshu Mishra commented on HIVE-22201:
[~jcamachorodriguez] can you please review
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932062#comment-16932062
]
Himanshu Mishra commented on HIVE-22201:
Failures are not related to patch, they passed on the first run and failed on
second, without any code change.
[testPartitionDiscoveryTransactionalTable|https://builds.apache.org/job/PreCommit-HIVE-Build/18590/testReport/org.apache.hadoop.hive.metastore/TestPartitionManagement/testPartitionDiscoveryTransactionalTable/]
[loadDataNonAcid2AcidConversion|https://builds.apache.org/job/PreCommit-HIVE-Build/18590/testReport/org.apache.hadoop.hive.ql/TestTxnLoadData/loadDataNonAcid2AcidConversion/]
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932047#comment-16932047
]
Hive QA commented on HIVE-22191:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 15 new + 566 unchanged - 47
fixed = 581 total (was 613) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
6s{color} | {color:red} ql generated 2 new + 1555 unchanged - 16 fixed = 1557
total (was 1571) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | tmp could be null and is guaranteed to be dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genColListRegex(String,
String, ASTNode, List, Set, RowResolver, RowResolver, Integer, RowResolver,
List, boolean) Dereferenced at SemanticAnalyzer.java:is guaranteed to be
dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genColListRegex(String,
String, ASTNode, List, Set, RowResolver, RowResolver, Integer, RowResolver,
List, boolean) Dereferenced at SemanticAnalyzer.java:[line 3727] |
| | Switch statement found in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processBoundary(ASTNode) where
default case is missing At SemanticAnalyzer.java:where default case is missing
At SemanticAnalyzer.java:[lines 14502-14512] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18629/dev-support/hive-personality.sh
|
| git revision | master / 5c83924 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18629/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18629/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18629/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for
[jira] [Commented] (HIVE-22211) Change maven phase to generate test sources
[
https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932029#comment-16932029
]
Hive QA commented on HIVE-22211:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980513/HIVE-22211.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18628/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18628/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18628/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980513 - PreCommit-HIVE-Build
> Change maven phase to generate test sources
> ---
>
> Key: HIVE-22211
> URL: https://issues.apache.org/jira/browse/HIVE-22211
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22211.01.patch
>
>
> Some protobuf files are generated in the wrong phase; so I get compile errors
> because they are not there for eclipse...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22211) Change maven phase to generate test sources
[
https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931998#comment-16931998
]
Hive QA commented on HIVE-22211:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18628/dev-support/hive-personality.sh
|
| git revision | master / 5c83924 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18628/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Change maven phase to generate test sources
> ---
>
> Key: HIVE-22211
> URL: https://issues.apache.org/jira/browse/HIVE-22211
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22211.01.patch
>
>
> Some protobuf files are generated in the wrong phase; so I get compile errors
> because they are not there for eclipse...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog
[ https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22213: --- Attachment: HIVE-22213.1.patch > TxnHander cleanupRecords should only clean records belonging to default > catalog > --- > > Key: HIVE-22213 > URL: https://issues.apache.org/jira/browse/HIVE-22213 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22213.1.patch > > > Currently it removes record for given database and given table without > checking for the catalog, as a result it can end up removing records when it > shouldn't. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog
[ https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22213: --- Attachment: (was: HIVE-22213.1.patch) > TxnHander cleanupRecords should only clean records belonging to default > catalog > --- > > Key: HIVE-22213 > URL: https://issues.apache.org/jira/browse/HIVE-22213 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > > Currently it removes record for given database and given table without > checking for the catalog, as a result it can end up removing records when it > shouldn't. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog
[ https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22213: --- Attachment: HIVE-22213.1.patch > TxnHander cleanupRecords should only clean records belonging to default > catalog > --- > > Key: HIVE-22213 > URL: https://issues.apache.org/jira/browse/HIVE-22213 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22213.1.patch > > > Currently it removes record for given database and given table without > checking for the catalog, as a result it can end up removing records when it > shouldn't. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog
[ https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22213: --- Status: Patch Available (was: Open) > TxnHander cleanupRecords should only clean records belonging to default > catalog > --- > > Key: HIVE-22213 > URL: https://issues.apache.org/jira/browse/HIVE-22213 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22213.1.patch > > > Currently it removes record for given database and given table without > checking for the catalog, as a result it can end up removing records when it > shouldn't. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog
[ https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-22213: -- > TxnHander cleanupRecords should only clean records belonging to default > catalog > --- > > Key: HIVE-22213 > URL: https://issues.apache.org/jira/browse/HIVE-22213 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > > Currently it removes record for given database and given table without > checking for the catalog, as a result it can end up removing records when it > shouldn't. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931988#comment-16931988
]
Hive QA commented on HIVE-22038:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980510/HIVE-22038.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingLeader.testHouseKeepingThreadExistence
(batchId=242)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18627/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18627/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18627/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980510 - PreCommit-HIVE-Build
> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch,
> HIVE-22038.02.patch, HIVE-22038.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931963#comment-16931963
]
Hive QA commented on HIVE-22038:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18627/dev-support/hive-personality.sh
|
| git revision | master / 5c83924 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18627/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch,
> HIVE-22038.02.patch, HIVE-22038.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931948#comment-16931948
]
Hive QA commented on HIVE-22201:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980509/HIVE-22201.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestPartitionManagement.testPartitionDiscoveryTransactionalTable
(batchId=223)
org.apache.hadoop.hive.ql.TestTxnLoadData.loadDataNonAcid2AcidConversion
(batchId=308)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18626/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18626/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18626/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980509 - PreCommit-HIVE-Build
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931932#comment-16931932
]
Hive QA commented on HIVE-22201:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18626/dev-support/hive-personality.sh
|
| git revision | master / 5c83924 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18626/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22210) Vectorization may reuse computation output columns involved in filtering
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931925#comment-16931925
]
Hive QA commented on HIVE-22210:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980501/HIVE-22210.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18625/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18625/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18625/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12980501/HIVE-22210.02.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980501 - PreCommit-HIVE-Build
> Vectorization may reuse computation output columns involved in filtering
>
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch, HIVE-22210.02.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22210) Vectorization may reuse computation output columns involved in filtering
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931924#comment-16931924
]
Hive QA commented on HIVE-22210:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980501/HIVE-22210.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16761 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18624/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18624/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18624/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980501 - PreCommit-HIVE-Build
> Vectorization may reuse computation output columns involved in filtering
>
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch, HIVE-22210.02.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22210) Vectorization may reuse computation output columns involved in filtering
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931912#comment-16931912
]
Hive QA commented on HIVE-22210:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 28 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 6 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18624/dev-support/hive-personality.sh
|
| git revision | master / 5c83924 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18624/yetus/whitespace-eol.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18624/yetus/whitespace-tabs.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18624/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization may reuse computation output columns involved in filtering
>
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch, HIVE-22210.02.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create
[jira] [Work started] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22042 started by John Sherman. --- > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931825#comment-16931825
]
Hive QA commented on HIVE-22195:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980489/HIVE-22195.7.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16763 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18623/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18623/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18623/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980489 - PreCommit-HIVE-Build
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
> URL: https://issues.apache.org/jira/browse/HIVE-22195
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch,
> HIVE-22195.4.patch, HIVE-22195.5.patch, HIVE-22195.6.patch, HIVE-22195.7.patch
>
>
> This could be useful in case when cluster is kerberized, but Zookeeper is not.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931800#comment-16931800
]
Hive QA commented on HIVE-22195:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
22s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
23s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 23s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} llap-client: The patch generated 1 new + 48 unchanged
- 2 fixed = 49 total (was 50) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
22s{color} | {color:red} service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18623/dev-support/hive-personality.sh
|
| git revision | master / 5c83924 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18623/yetus/patch-mvninstall-service.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18623/yetus/patch-compile-service.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18623/yetus/patch-compile-service.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18623/yetus/diff-checkstyle-llap-client.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18623/yetus/patch-findbugs-service.txt
|
| modules | C: common llap-client standalone-metastore/metastore-server service
U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18623/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
>
[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931758#comment-16931758
]
Hive QA commented on HIVE-22202:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980478/HIVE-22202.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16760 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18622/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18622/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18622/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980478 - PreCommit-HIVE-Build
> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch,
> HIVE-22202.3.patch
>
>
> Remove unnecessary code
> * remove unnecessary/unused variables
> * remove unused private methods
> * remove unnecessary throws declarations
> * remove unnecessary conditionals that are guaranteed to always give the
> same result
> * simplify collection handling
> ** use member sort() instead of Collections.sort()
> ** use addAll() instead of adding one-by-one in a loop
> * remove Javadoc that doesn't give additional information
> * use try with resource where possible
> * remove unnecessary explicit type arguments
> * remove redundant warning suppression
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Status: Open (was: Patch Available) > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch, HIVE-20113.7.patch, > HIVE-20113.8.patch, HIVE-20113.9.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Status: Patch Available (was: Open) > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch, HIVE-20113.7.patch, > HIVE-20113.8.patch, HIVE-20113.9.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Attachment: HIVE-20113.9.patch > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch, HIVE-20113.7.patch, > HIVE-20113.8.patch, HIVE-20113.9.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=313853=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313853 ] ASF GitHub Bot logged work on HIVE-22170: - Author: ASF GitHub Bot Created on: 17/Sep/19 18:45 Start Date: 17/Sep/19 18:45 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #764: HIVE-22170 URL: https://github.com/apache/hive/pull/764 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 313853) Time Spent: 1h (was: 50m) > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch, HIVE-22170.04.patch, HIVE-22170.05.patch, > HIVE-22170.06.patch > > Time Spent: 1h > Remaining Estimate: 0h > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22170: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch, HIVE-22170.04.patch, HIVE-22170.05.patch, > HIVE-22170.06.patch > > Time Spent: 1h > Remaining Estimate: 0h > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931742#comment-16931742 ] Vineet Garg commented on HIVE-22170: +1 > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch, HIVE-22170.04.patch, HIVE-22170.05.patch, > HIVE-22170.06.patch > > Time Spent: 50m > Remaining Estimate: 0h > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931734#comment-16931734
]
Hive QA commented on HIVE-22202:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
14s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} standalone-metastore/metastore-server: The patch
generated 0 new + 347 unchanged - 64 fixed = 347 total (was 411) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
15s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 170 unchanged - 8 fixed = 170 total (was 178) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18622/dev-support/hive-personality.sh
|
| git revision | master / 8055650 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18622/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch,
> HIVE-22202.3.patch
>
>
> Remove unnecessary code
> * remove unnecessary/unused variables
> * remove unused private methods
> * remove unnecessary throws declarations
> * remove unnecessary conditionals that are guaranteed to always give the
> same result
> * simplify collection handling
> ** use member sort() instead of Collections.sort()
> ** use addAll() instead of adding one-by-one in a loop
> * remove Javadoc that doesn't give additional information
> * use try with resource where possible
> * remove unnecessary explicit type arguments
> * remove redundant warning suppression
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21875) Implement drop partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931639#comment-16931639
]
Hive QA commented on HIVE-21875:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980476/HIVE-21875.06.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16832 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18621/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18621/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18621/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980476 - PreCommit-HIVE-Build
> Implement drop partition related methods on temporary tables
>
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch,
> HIVE-21875.03.patch, HIVE-21875.04.patch, HIVE-21875.05.patch,
> HIVE-21875.06.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, boolean deleteData);
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, PartitionDropOptions options);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, PartitionDropOptions options);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> boolean dropPartition(String db_name, String tbl_name, String name, boolean
> deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name, String
> name, boolean deleteData){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.5.patch
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21875) Implement drop partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931592#comment-16931592
]
Hive QA commented on HIVE-21875:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 1572 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} standalone-metastore/metastore-server: The patch
generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18621/dev-support/hive-personality.sh
|
| git revision | master / 8055650 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18621/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement drop partition related methods on temporary tables
>
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch,
> HIVE-21875.03.patch, HIVE-21875.04.patch, HIVE-21875.05.patch,
> HIVE-21875.06.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, boolean deleteData);
> boolean dropPartition(String db_name, String tbl_name, List
>
[jira] [Updated] (HIVE-22212) implement append partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22212:
-
Component/s: Hive
> implement append partition related methods on temporary tables
> --
>
> Key: HIVE-22212
> URL: https://issues.apache.org/jira/browse/HIVE-22212
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
>
> The following methods must be implemented in SessionHiveMetastoreClient, in
> order to support partition append on temporary tables:
> {code:java}
> Partition appendPartition(String dbName, String tableName, List
> partVals)
> throws InvalidObjectException, AlreadyExistsException, MetaException,
> TException;
> Partition appendPartition(String catName, String dbName, String tableName,
> List partVals)
> throws InvalidObjectException, AlreadyExistsException, MetaException,
> TException;
> Partition appendPartition(String dbName, String tableName, String name)
> throws InvalidObjectException, AlreadyExistsException, MetaException,
> TException;
> Partition appendPartition(String catName, String dbName, String tableName,
> String name)
> throws InvalidObjectException, AlreadyExistsException, MetaException,
> TException;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22212) implement append partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter reassigned HIVE-22212:
> implement append partition related methods on temporary tables
> --
>
> Key: HIVE-22212
> URL: https://issues.apache.org/jira/browse/HIVE-22212
> Project: Hive
> Issue Type: Sub-task
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
>
> The following methods must be implemented in SessionHiveMetastoreClient, in
> order to support partition append on temporary tables:
> {code:java}
> Partition appendPartition(String dbName, String tableName, List
> partVals)
> throws InvalidObjectException, AlreadyExistsException, MetaException,
> TException;
> Partition appendPartition(String catName, String dbName, String tableName,
> List partVals)
> throws InvalidObjectException, AlreadyExistsException, MetaException,
> TException;
> Partition appendPartition(String dbName, String tableName, String name)
> throws InvalidObjectException, AlreadyExistsException, MetaException,
> TException;
> Partition appendPartition(String catName, String dbName, String tableName,
> String name)
> throws InvalidObjectException, AlreadyExistsException, MetaException,
> TException;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21884.12.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21884.01.patch, HIVE-21884.02.patch, > HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch, > HIVE-21884.10.patch, HIVE-21884.11.patch, HIVE-21884.12.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931491#comment-16931491
]
Ana Jalba edited comment on HIVE-21508 at 9/17/19 2:04 PM:
---
[~pvary]: I'm guessing I can't check locally if the ptests fail with/without
the patch, right?
What I can say though, is that locally, the tests pass, except for some tests
in TestRpc, ThriftCliServiceTestWithCookie, TestLdapAtnProviderWithMiniDS, and
TestHiveCli, but this happens without the patch too.
was (Author: ananamj):
[~pvary]: I'm guessing I can't check locally if the ptests fail with/without
the patch, right?
What I can say though, is that locally, the tests pass, except for some tests
in TestRpc, ThriftCliServiceTestWithCookie, TestLdapAtnProviderWithMiniDS, and
TestHiveCli but this happens without the patch too.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch,
> HIVE-21508.5.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931491#comment-16931491
]
Ana Jalba commented on HIVE-21508:
--
[~pvary]: I'm guessing I can't check locally if the ptests fail with/without
the patch, right?
What I can say though, is that locally, the tests pass, except for some tests
in TestRpc, ThriftCliServiceTestWithCookie, TestLdapAtnProviderWithMiniDS, and
TestHiveCli but this happens without the patch too.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch,
> HIVE-21508.5.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22211) Change maven phase to generate test sources
[ https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931468#comment-16931468 ] Laszlo Bodor commented on HIVE-22211: - +1 > Change maven phase to generate test sources > --- > > Key: HIVE-22211 > URL: https://issues.apache.org/jira/browse/HIVE-22211 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22211.01.patch > > > Some protobuf files are generated in the wrong phase; so I get compile errors > because they are not there for eclipse... -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Status: In Progress (was: Patch Available)
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Status: Patch Available (was: In Progress)
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Attachment: HIVE-22191.2.patch
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.2.patch, HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Attachment: (was: HIVE-22191.2.patch)
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22191) Simplify SemanticAnalyzer by removing unused code
[
https://issues.apache.org/jira/browse/HIVE-22191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22191:
--
Attachment: HIVE-22191.2.patch
> Simplify SemanticAnalyzer by removing unused code
> -
>
> Key: HIVE-22191
> URL: https://issues.apache.org/jira/browse/HIVE-22191
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22191.patch
>
>
> Simplify {{SemanticAnalyzer}} by:
> - Remove dead code
> - Simplify returning statements
> - Use interfaces types for parameters/fields/variables where it is
> straightforward to migrate
> - Make visibility stricter where it is possible
> - Check logging to use parametrised logging
> - Removing unnecessary keywords (e.g.: {{static}})
> - Some code part could be simplified by using Java 8 features
> I think this is crucial step as this class already contains 15000+ lines of
> code which is screaming for splitting into more reasonable classes.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22059:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
pushed to master. Thank you [~abstractdog] for reviewing the changes!
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch, HIVE-22059.04.patch,
> HIVE-22059.05.patch, HIVE-22059.06.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.databind.ObjectMapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 19 more
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1 (state=08S01,code=2)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22065) Code cleanup around org.apache.hadoop.hive.ql.exec.tez.RecordProcessor
[ https://issues.apache.org/jira/browse/HIVE-22065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22065: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you [~isuller]! > Code cleanup around org.apache.hadoop.hive.ql.exec.tez.RecordProcessor > -- > > Key: HIVE-22065 > URL: https://issues.apache.org/jira/browse/HIVE-22065 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22065.1.patch, HIVE-22065.2.patch > > > I addressed the following issues: > - replace explicit generic definitions with diamond operator where applicable > - fix field visibility > - use lambda expression instead of instantiating a custom interface > implementation > - remove unnecessary null checks -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931409#comment-16931409
]
Hive QA commented on HIVE-21884:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 6m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
35s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
12s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
9s{color} | {color:blue} ql in master has 1572 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} itests/util in master has 51 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
46s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} standalone-metastore: The patch generated 19 new +
1743 unchanged - 15 fixed = 1762 total (was 1758) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 1 new + 493 unchanged - 0 fixed = 494 total (was 493) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} common: The patch generated 2 new + 367 unchanged - 0
fixed = 369 total (was 367) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch service-rpc passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
30s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 18 new + 1250 unchanged - 15 fixed = 1268 total (was 1265) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 59 new + 710 unchanged - 0
fixed = 769 total (was 710) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} service: The patch generated 0 new + 38 unchanged -
1 fixed = 38 total (was 39) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
34s{color} | {color:red} root: The patch generated 97 new + 2905 unchanged - 16
fixed = 3002 total (was 2921) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:red}-1{color}
[jira] [Updated] (HIVE-22211) Change maven phase to generate test sources
[ https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22211: Attachment: HIVE-22211.01.patch > Change maven phase to generate test sources > --- > > Key: HIVE-22211 > URL: https://issues.apache.org/jira/browse/HIVE-22211 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22211.01.patch > > > Some protobuf files are generated in the wrong phase; so I get compile errors > because they are not there for eclipse... -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22211) Change maven phase to generate test sources
[ https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22211: Status: Patch Available (was: Open) > Change maven phase to generate test sources > --- > > Key: HIVE-22211 > URL: https://issues.apache.org/jira/browse/HIVE-22211 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22211.01.patch > > > Some protobuf files are generated in the wrong phase; so I get compile errors > because they are not there for eclipse... -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22211) Change maven phase to generate test sources
[ https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-22211: --- > Change maven phase to generate test sources > --- > > Key: HIVE-22211 > URL: https://issues.apache.org/jira/browse/HIVE-22211 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22211.01.patch > > > Some protobuf files are generated in the wrong phase; so I get compile errors > because they are not there for eclipse... -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931373#comment-16931373
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980465/HIVE-21884.11.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 16796 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables_sysdb]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join_hash]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_reduce_side]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_aggregate_without_gby]
(batchId=176)
org.apache.hadoop.hive.metastore.tools.schematool.TestSchemaToolForMetastore.testMetastoreDbPropertiesAfterUpgrade
(batchId=227)
org.apache.hadoop.hive.metastore.tools.schematool.TestSchemaToolForMetastore.testValidateSchemaTables
(batchId=227)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths
(batchId=274)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testCancelRenewTokenFlow
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testConnection
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValid (batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValidNeg
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeProxyAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeTokenAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNoKrbSASLTokenAuthNeg
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLFullNameAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testProxyAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testRenewDelegationToken
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testTokenAuth
(batchId=298)
org.apache.hive.minikdc.TestSSLWithMiniKdc.testConnection (batchId=298)
org.apache.hive.service.cli.thrift.TestThriftCLIServiceSecurity.testPasswordNotInLogs
(batchId=279)
org.apache.hive.service.server.TestHS2HttpServerPamConfiguration.testPamCorrectConfiguration
(batchId=238)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18620/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18620/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18620/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980465 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21884.01.patch, HIVE-21884.02.patch,
> HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch,
> HIVE-21884.10.patch, HIVE-21884.11.patch, Scheduled queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[ https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22038: Attachment: HIVE-22038.02.patch > Fix memory related sideeffects of opening/closing sessions > -- > > Key: HIVE-22038 > URL: https://issues.apache.org/jira/browse/HIVE-22038 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch, > HIVE-22038.02.patch, HIVE-22038.02.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Attachment: HIVE-22201.02.patch
Status: Patch Available (was: Open)
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Status: Open (was: Patch Available)
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Status: Patch Available (was: Open)
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected
[
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Mishra updated HIVE-22201:
---
Status: Open (was: Patch Available)
> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22201.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1,
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22210) Vectorization may reuse computation output columns involved in filtering
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22210:
Attachment: HIVE-22210.02.patch
> Vectorization may reuse computation output columns involved in filtering
>
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch, HIVE-22210.02.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22210) Vectorization may reuse computation output columns involved in filtering
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931288#comment-16931288
]
Laszlo Bodor commented on HIVE-22210:
-
+1, pending tests
> Vectorization may reuse computation output columns involved in filtering
>
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22210) Vectorization may reuse computation output columns involved in filtering
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22210:
Summary: Vectorization may reuse computation output columns involved in
filtering (was: Incorrect results in some cases with vectorization)
> Vectorization may reuse computation output columns involved in filtering
>
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22210) Incorrect results in some cases with vectorization
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931273#comment-16931273
]
Zoltan Haindrich commented on HIVE-22210:
-
I suppose IdentityExpression's output may not be freed; as its directly relays
the column...there were already a tweak for this:
https://github.com/apache/hive/blob/62f0c3fe89d7112d2cbc5c033290492be98a5556/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java#L2358
but in case a filter expression was created this was missing...
> Incorrect results in some cases with vectorization
> --
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22210) Incorrect results in some cases with vectorization
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931274#comment-16931274
]
Zoltan Haindrich commented on HIVE-22210:
-
I suppose IdentityExpression's output may not be freed; as its directly relays
the column...there were already a tweak for this:
https://github.com/apache/hive/blob/62f0c3fe89d7112d2cbc5c033290492be98a5556/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java#L2358
but in case a filter expression was created this was missing...
> Incorrect results in some cases with vectorization
> --
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22210) Incorrect results in some cases with vectorization
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22210:
Status: Patch Available (was: Open)
> Incorrect results in some cases with vectorization
> --
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22210) Incorrect results in some cases with vectorization
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22210:
Attachment: HIVE-22210.01.patch
> Incorrect results in some cases with vectorization
> --
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22210.01.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[ https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22195: -- Attachment: HIVE-22195.7.patch > Configure authentication type for Zookeeper when different from the default > cluster wide > > > Key: HIVE-22195 > URL: https://issues.apache.org/jira/browse/HIVE-22195 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch, > HIVE-22195.4.patch, HIVE-22195.5.patch, HIVE-22195.6.patch, HIVE-22195.7.patch > > > This could be useful in case when cluster is kerberized, but Zookeeper is not. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22197) Common Merge join throwing class cast exception
[
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931207#comment-16931207
]
Hive QA commented on HIVE-22197:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980460/HIVE-22197.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16761 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18619/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18619/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18619/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980460 - PreCommit-HIVE-Build
> Common Merge join throwing class cast exception
>
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch,
> HIVE-22197.03.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed
> and stored in the writable format, but the object inspector is initialized to
> default. So when join operator is fetching the data from dummy store
> operator, its getting the OI is Long and the row as LongWritable. This is
> causing the class cast exception.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22210) Incorrect results in some cases with vectorization
[
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-22210:
---
> Incorrect results in some cases with vectorization
> --
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>
> running the following test with TestMiniLlapLocalCliDriver leads to an
> unexpected results; the coalesce calculated inside the subquery has a value
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
> SELECT
> a5.int_col,
> 922 as expected,
> COALESCE(a5.int_col, a5.aa) as expected2,
> a5.int_col_3 as reality
> FROMu_table_19 a1
> FULL OUTER JOIN
> (
>SELECT a2.int_col_5 AS int_col,
> a2.smallint_col_22 as aa,
> COALESCE(a2.int_col_5, a2.smallint_col_22) AS
> int_col_3
>FROM u_table_4 a2
> ) a5
> ON (
> a1.bigint_col_18) = (a5.int_col_3)
> INNER JOIN
> (
> SELECT a3.bigint_col_3
> AS int_col,
> Cast (COALESCE(a3.bigint_col_3,
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT)
> AS int_col_3
> FROM u_table_7 a3
> WHEREbigint_col_3=571
> ) a4
> ON (a1.int_col_19=5)
> OR ((a5.int_col_3) IN (a4.int_col, 10))
> where
> a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22197) Common Merge join throwing class cast exception
[
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931164#comment-16931164
]
Hive QA commented on HIVE-22197:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
55s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 1572 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 29 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18619/dev-support/hive-personality.sh
|
| git revision | master / 62f0c3f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18619/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18619/yetus/whitespace-tabs.txt
|
| modules | C: shims/0.23 ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18619/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Common Merge join throwing class cast exception
>
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch,
> HIVE-22197.03.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed
> and stored in the writable format, but the object inspector is initialized to
> default. So when join operator is fetching the data from dummy store
> operator, its getting the OI is Long and the row as LongWritable. This is
> causing the class cast exception.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Status: In Progress (was: Patch Available) > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch, > HIVE-22202.3.patch > > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Status: Patch Available (was: In Progress) > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch, > HIVE-22202.3.patch > > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Attachment: HIVE-22202.3.patch > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch, > HIVE-22202.3.patch > > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21875) Implement drop partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-21875:
-
Attachment: HIVE-21875.06.patch
> Implement drop partition related methods on temporary tables
>
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch,
> HIVE-21875.03.patch, HIVE-21875.04.patch, HIVE-21875.05.patch,
> HIVE-21875.06.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, boolean deleteData);
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, PartitionDropOptions options);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, PartitionDropOptions options);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> boolean dropPartition(String db_name, String tbl_name, String name, boolean
> deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name, String
> name, boolean deleteData){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931146#comment-16931146
]
Laszlo Bodor commented on HIVE-22059:
-
+1
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch, HIVE-22059.04.patch,
> HIVE-22059.05.patch, HIVE-22059.06.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.databind.ObjectMapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 19 more
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1 (state=08S01,code=2)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931143#comment-16931143
]
Peter Vary commented on HIVE-21508:
---
[~ananamj]: Forget about yetus in this case. Concentrate on the Ptest (2nd
comment).
For me it seems that PTest checks out 3.1 branch:
{code}
2019-09-13 14:43:02,157 INFO [StreamReader-[bash
/data/hiveptest/working/scratch/source-prep.sh]]
LocalCommand$CollectLogPolicy.handleOutput:101 + git checkout branch-3.1
{code}
If it is so, then we might try to run ptest without a real change in a patch to
check if the 126 failining tests are unrelated.
Thanks,
Peter
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch,
> HIVE-21508.5.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931136#comment-16931136
]
Ana Jalba commented on HIVE-21508:
--
[~pvary] Any advice on how I could fix this or should I just move on with
branch-3?
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.3.branch-2.patch, HIVE-21508.4.branch-3.1.patch,
> HIVE-21508.5.branch-3.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22182) SemanticAnalyzer populates map which is not used at all
[
https://issues.apache.org/jira/browse/HIVE-22182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931129#comment-16931129
]
Laszlo Bodor commented on HIVE-22182:
-
pushed to master, thanks [~adombi] for the patch!
> SemanticAnalyzer populates map which is not used at all
> ---
>
> Key: HIVE-22182
> URL: https://issues.apache.org/jira/browse/HIVE-22182
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22182.patch
>
>
> The {{SemanticAnalyzer}} class has two map fields which are populated but
> never queried:
> * {{topToTable}}
> * {{topToTableProps}}
> Last usage was removed in HIVE-11752.
> h3. Purpose of this task
> - Remove unused {{topToTable}} and {{topToTableProps}} fields
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22182) SemanticAnalyzer populates map which is not used at all
[
https://issues.apache.org/jira/browse/HIVE-22182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-22182:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> SemanticAnalyzer populates map which is not used at all
> ---
>
> Key: HIVE-22182
> URL: https://issues.apache.org/jira/browse/HIVE-22182
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22182.patch
>
>
> The {{SemanticAnalyzer}} class has two map fields which are populated but
> never queried:
> * {{topToTable}}
> * {{topToTableProps}}
> Last usage was removed in HIVE-11752.
> h3. Purpose of this task
> - Remove unused {{topToTable}} and {{topToTableProps}} fields
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22208) Column name with reserved keyword is unescaped when query including join on table with mask column is re-written
[
https://issues.apache.org/jira/browse/HIVE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi reassigned HIVE-22208:
---
Assignee: Jesus Camacho Rodriguez
> Column name with reserved keyword is unescaped when query including join on
> table with mask column is re-written
>
>
> Key: HIVE-22208
> URL: https://issues.apache.org/jira/browse/HIVE-22208
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
>
> Join query involving table with mask column and other having reserved
> keyword as column name fails with SemanticException during parsing re-written
> query :
> Original Query :
> {code:java}
> select a.`date`, b.nm
> from sample_keyword a
> join sample_mask b
> on b.id = a.id;
> {code}
> Re-written Query :
>
> {code:java}
> select a.date, b.nm
> from sample_keyword a
> join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
> BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
> `default`.`sample_mask` )`b`
> on b.id = a.id;
> {code}
> Re-written query does not have escape quotes for date column which cause
> SemanticException while parsing :
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near 'a' '.' 'date' in selection target
>
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
> SemanticAnalyzer.java:12084)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
> SemanticAnalyzer.java:12298)
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
> CalcitePlanner.java:360)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
> BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21884.11.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21884.01.patch, HIVE-21884.02.patch, > HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch, > HIVE-21884.10.patch, HIVE-21884.11.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.2#803003)
