[jira] [Resolved] (HIVE-26606) Expose failover states in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-26606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshal Patel resolved HIVE-26606. -- Resolution: Fixed > Expose failover states in replication metrics > - > > Key: HIVE-26606 > URL: https://issues.apache.org/jira/browse/HIVE-26606 > Project: Hive > Issue Type: Improvement >Reporter: Teddy Choi >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Attachments: > HIVE-26606__Expose_failover_states_in_replication_metrics1.patch > > Time Spent: 2h > Remaining Estimate: 0h > > Expose the state of failover in replication metrics, -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26595) Expose failover states in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-26595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685740#comment-17685740 ] Harshal Patel commented on HIVE-26595: -- Fixed as part of HIVE-26932 > Expose failover states in replication metrics > - > > Key: HIVE-26595 > URL: https://issues.apache.org/jira/browse/HIVE-26595 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Priority: Major > > Expose the state of failover in replication metrics -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-26595) Expose failover states in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-26595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshal Patel reassigned HIVE-26595: Assignee: Harshal Patel > Expose failover states in replication metrics > - > > Key: HIVE-26595 > URL: https://issues.apache.org/jira/browse/HIVE-26595 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > > Expose the state of failover in replication metrics -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26595) Expose failover states in replication metrics
[ https://issues.apache.org/jira/browse/HIVE-26595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshal Patel resolved HIVE-26595. -- Resolution: Duplicate > Expose failover states in replication metrics > - > > Key: HIVE-26595 > URL: https://issues.apache.org/jira/browse/HIVE-26595 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > > Expose the state of failover in replication metrics -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26932) Correct stage name value in replication_metrics.progress column in replication_metrics table
[ https://issues.apache.org/jira/browse/HIVE-26932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshal Patel resolved HIVE-26932. -- Resolution: Fixed > Correct stage name value in replication_metrics.progress column in > replication_metrics table > > > Key: HIVE-26932 > URL: https://issues.apache.org/jira/browse/HIVE-26932 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > To improve diagnostic capability from Source to backup replication, update > replication_metrics table by adding pre_optimized_bootstrap in progress bar > in case of optimized bootstrap first cycle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-26932) Correct stage name value in replication_metrics.progress column in replication_metrics table
[ https://issues.apache.org/jira/browse/HIVE-26932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685738#comment-17685738 ] Harshal Patel commented on HIVE-26932: -- Merged as part of https://github.com/apache/hive/pull/3990 > Correct stage name value in replication_metrics.progress column in > replication_metrics table > > > Key: HIVE-26932 > URL: https://issues.apache.org/jira/browse/HIVE-26932 > Project: Hive > Issue Type: Improvement >Reporter: Harshal Patel >Assignee: Harshal Patel >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > To improve diagnostic capability from Source to backup replication, update > replication_metrics table by adding pre_optimized_bootstrap in progress bar > in case of optimized bootstrap first cycle. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26920) Add-new-view-in-sys-db-to-capture-failover-and-failback-metrics
[ https://issues.apache.org/jira/browse/HIVE-26920?focusedWorklogId=844252=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844252 ] ASF GitHub Bot logged work on HIVE-26920: - Author: ASF GitHub Bot Created on: 08/Feb/23 07:39 Start Date: 08/Feb/23 07:39 Worklog Time Spent: 10m Work Description: pudidic commented on PR #4023: URL: https://github.com/apache/hive/pull/4023#issuecomment-1422164784 +1. LGTM. Issue Time Tracking --- Worklog Id: (was: 844252) Time Spent: 50m (was: 40m) > Add-new-view-in-sys-db-to-capture-failover-and-failback-metrics > --- > > Key: HIVE-26920 > URL: https://issues.apache.org/jira/browse/HIVE-26920 > Project: Hive > Issue Type: New Feature > Components: HiveServer2 >Reporter: Amit Saonerkar >Assignee: Amit Saonerkar >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Corresponding to CDPD-46702 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27049) Iceberg: Provide current snapshot version in show-create-table
[ https://issues.apache.org/jira/browse/HIVE-27049?focusedWorklogId=844249=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844249 ] ASF GitHub Bot logged work on HIVE-27049: - Author: ASF GitHub Bot Created on: 08/Feb/23 07:26 Start Date: 08/Feb/23 07:26 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4036: URL: https://github.com/apache/hive/pull/4036#issuecomment-1422143814 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4036) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4036=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4036=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844249) Time Spent: 50m (was: 40m) > Iceberg: Provide current snapshot version in show-create-table > -- > > Key: HIVE-27049 > URL: https://issues.apache.org/jira/browse/HIVE-27049 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: Rajesh Balamohan >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > It will be helpful to show "current snapshot" id in "show create table" > statement. This will help in easier debugging. Otherwise, user has to > explicitly query the metadata or read the JSON file to get this info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27059) Wrong object inspector will be created when use collect_list and disable map-side aggregation
[
https://issues.apache.org/jira/browse/HIVE-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27059:
--
Labels: pull-request-available (was: )
> Wrong object inspector will be created when use collect_list and disable
> map-side aggregation
> -
>
> Key: HIVE-27059
> URL: https://issues.apache.org/jira/browse/HIVE-27059
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 2.3.8, 3.1.3, 4.0.0-alpha-2
> Environment:
>Reporter: Genmao Yu
>Assignee: Genmao Yu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Query will fail when use collect_list (or collect_set) and disable map-side
> aggregationg:
> {code:java}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryMap cannot be cast to
> java.util.Map
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardMapObjectInspector.getMap(StandardMapObjectInspector.java:85)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:437)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:362)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.putIntoCollection(GenericUDAFMkCollectionEvaluator.java:154)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.iterate(GenericUDAFMkCollectionEvaluator.java:120)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:192)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:638)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:877)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:721)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:787)
> {code}
> To reproduce this issue:
> {code:sql}
> create table tb1 (a int, b string, c string);
> insert into tb1 values (1, "100", "101");
> insert into tb1 values (1, "102", "103");
> insert into tb1 values (2, "200", "201");
> set hive.map.aggr=false;
> select a, collect_list(map("b",b,"c",c)) as col1 from tb1 group by a;
> select a, collect_set(array(b, c)) as col1 from tb1 group by a;
> {code}
> To work around this issue:
> {code:sql}
> set hive.map.aggr=true;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-27059) Wrong object inspector will be created when use collect_list and disable map-side aggregation
[
https://issues.apache.org/jira/browse/HIVE-27059?focusedWorklogId=844242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844242
]
ASF GitHub Bot logged work on HIVE-27059:
-
Author: ASF GitHub Bot
Created on: 08/Feb/23 07:12
Start Date: 08/Feb/23 07:12
Worklog Time Spent: 10m
Work Description: uncleGen opened a new pull request, #4042:
URL: https://github.com/apache/hive/pull/4042
### What changes were proposed in this pull request?
Query will fail when use collect_list (or collect_set) and disable map-side
aggregationg:
```
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryMap cannot be cast to
java.util.Map
at
org.apache.hadoop.hive.serde2.objectinspector.StandardMapObjectInspector.getMap(StandardMapObjectInspector.java:85)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:437)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:362)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.putIntoCollection(GenericUDAFMkCollectionEvaluator.java:154)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.iterate(GenericUDAFMkCollectionEvaluator.java:120)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:192)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:638)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:877)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:721)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:787)
```
To reproduce this issue:
```
create table tb1 (a int, b string, c string);
insert into tb1 values (1, "100", "101");
insert into tb1 values (1, "102", "103");
insert into tb1 values (2, "200", "201");
set hive.map.aggr=false;
select a, collect_list(map("b",b,"c",c)) as col1 from tb1 group by a;
select a, collect_set(array(b, c)) as col1 from tb1 group by a;
```
### Why are the changes needed?
Wrong object inspector will be created when use collect_list and disable
map-side aggregation.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
UT and manual test
Issue Time Tracking
---
Worklog Id: (was: 844242)
Remaining Estimate: 0h
Time Spent: 10m
> Wrong object inspector will be created when use collect_list and disable
> map-side aggregation
> -
>
> Key: HIVE-27059
> URL: https://issues.apache.org/jira/browse/HIVE-27059
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 2.3.8, 3.1.3, 4.0.0-alpha-2
> Environment:
>Reporter: Genmao Yu
>Assignee: Genmao Yu
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Query will fail when use collect_list (or collect_set) and disable map-side
> aggregationg:
> {code:java}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryMap cannot be cast to
> java.util.Map
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardMapObjectInspector.getMap(StandardMapObjectInspector.java:85)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:437)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:362)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.putIntoCollection(GenericUDAFMkCollectionEvaluator.java:154)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.iterate(GenericUDAFMkCollectionEvaluator.java:120)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:192)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:638)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:877)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:721)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:787)
> {code}
> To reproduce this issue:
> {code:sql}
> create table tb1 (a int, b string,
[jira] [Assigned] (HIVE-27059) Wrong object inspector will be created when use collect_list and disable map-side aggregation
[
https://issues.apache.org/jira/browse/HIVE-27059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Genmao Yu reassigned HIVE-27059:
> Wrong object inspector will be created when use collect_list and disable
> map-side aggregation
> -
>
> Key: HIVE-27059
> URL: https://issues.apache.org/jira/browse/HIVE-27059
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0-alpha-2, 3.1.3, 2.3.8
> Environment:
>Reporter: Genmao Yu
>Assignee: Genmao Yu
>Priority: Major
>
> Query will fail when use collect_list (or collect_set) and disable map-side
> aggregationg:
> {code:java}
> Caused by: java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryMap cannot be cast to
> java.util.Map
> at
> org.apache.hadoop.hive.serde2.objectinspector.StandardMapObjectInspector.getMap(StandardMapObjectInspector.java:85)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:437)
> at
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:362)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.putIntoCollection(GenericUDAFMkCollectionEvaluator.java:154)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.iterate(GenericUDAFMkCollectionEvaluator.java:120)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:192)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:638)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:877)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:721)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:787)
> {code}
> To reproduce this issue:
> {code:sql}
> create table tb1 (a int, b string, c string);
> insert into tb1 values (1, "100", "101");
> insert into tb1 values (1, "102", "103");
> insert into tb1 values (2, "200", "201");
> set hive.map.aggr=false;
> select a, collect_list(map("b",b,"c",c)) as col1 from tb1 group by a;
> select a, collect_set(array(b, c)) as col1 from tb1 group by a;
> {code}
> To work around this issue:
> {code:sql}
> set hive.map.aggr=true;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26920) Add-new-view-in-sys-db-to-capture-failover-and-failback-metrics
[ https://issues.apache.org/jira/browse/HIVE-26920?focusedWorklogId=844237=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844237 ] ASF GitHub Bot logged work on HIVE-26920: - Author: ASF GitHub Bot Created on: 08/Feb/23 06:11 Start Date: 08/Feb/23 06:11 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4023: URL: https://github.com/apache/hive/pull/4023#issuecomment-1422070087 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4023) [](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4023=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4023=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4023=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4023=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4023=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4023=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844237) Time Spent: 40m (was: 0.5h) > Add-new-view-in-sys-db-to-capture-failover-and-failback-metrics > --- > > Key: HIVE-26920 > URL: https://issues.apache.org/jira/browse/HIVE-26920 > Project: Hive > Issue Type: New Feature > Components: HiveServer2 >Reporter: Amit Saonerkar >Assignee: Amit Saonerkar >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Corresponding to CDPD-46702 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27058) Backport of HIVE-24316: Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-27058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diksha updated HIVE-27058: -- Labels: hive-3.2.0-must (was: ) > Backport of HIVE-24316: Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1 > - > > Key: HIVE-27058 > URL: https://issues.apache.org/jira/browse/HIVE-27058 > Project: Hive > Issue Type: Sub-task >Reporter: Diksha >Assignee: Diksha >Priority: Major > Labels: hive-3.2.0-must > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27058) Backport of HIVE-24316: Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1
[ https://issues.apache.org/jira/browse/HIVE-27058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diksha reassigned HIVE-27058: - > Backport of HIVE-24316: Upgrade ORC from 1.5.6 to 1.5.8 in branch-3.1 > - > > Key: HIVE-27058 > URL: https://issues.apache.org/jira/browse/HIVE-27058 > Project: Hive > Issue Type: Sub-task >Reporter: Diksha >Assignee: Diksha >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27016) Invoke optional output committer in TezProcessor
[ https://issues.apache.org/jira/browse/HIVE-27016?focusedWorklogId=844230=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844230 ] ASF GitHub Bot logged work on HIVE-27016: - Author: ASF GitHub Bot Created on: 08/Feb/23 05:20 Start Date: 08/Feb/23 05:20 Worklog Time Spent: 10m Work Description: sunchao commented on PR #4013: URL: https://github.com/apache/hive/pull/4013#issuecomment-1422032002 Merged, thanks Issue Time Tracking --- Worklog Id: (was: 844230) Time Spent: 1h 20m (was: 1h 10m) > Invoke optional output committer in TezProcessor > > > Key: HIVE-27016 > URL: https://issues.apache.org/jira/browse/HIVE-27016 > Project: Hive > Issue Type: Improvement > Components: Query Processor, StorageHandler >Affects Versions: 3.1.3 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > This is backport HIVE-24629 and HIVE-24867, so StorageHandler with their own > OutputCommitter run in tez. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27016) Invoke optional output committer in TezProcessor
[ https://issues.apache.org/jira/browse/HIVE-27016?focusedWorklogId=844229=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844229 ] ASF GitHub Bot logged work on HIVE-27016: - Author: ASF GitHub Bot Created on: 08/Feb/23 05:20 Start Date: 08/Feb/23 05:20 Worklog Time Spent: 10m Work Description: sunchao merged PR #4013: URL: https://github.com/apache/hive/pull/4013 Issue Time Tracking --- Worklog Id: (was: 844229) Time Spent: 1h 10m (was: 1h) > Invoke optional output committer in TezProcessor > > > Key: HIVE-27016 > URL: https://issues.apache.org/jira/browse/HIVE-27016 > Project: Hive > Issue Type: Improvement > Components: Query Processor, StorageHandler >Affects Versions: 3.1.3 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > This is backport HIVE-24629 and HIVE-24867, so StorageHandler with their own > OutputCommitter run in tez. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27016) Invoke optional output committer in TezProcessor
[ https://issues.apache.org/jira/browse/HIVE-27016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved HIVE-27016. - Fix Version/s: 3.2.0 Resolution: Fixed > Invoke optional output committer in TezProcessor > > > Key: HIVE-27016 > URL: https://issues.apache.org/jira/browse/HIVE-27016 > Project: Hive > Issue Type: Improvement > Components: Query Processor, StorageHandler >Affects Versions: 3.1.3 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > This is backport HIVE-24629 and HIVE-24867, so StorageHandler with their own > OutputCommitter run in tez. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26961) Fix improper replication metric count when hive.repl.filter.transactions is set to true.
[ https://issues.apache.org/jira/browse/HIVE-26961?focusedWorklogId=844228=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844228 ] ASF GitHub Bot logged work on HIVE-26961: - Author: ASF GitHub Bot Created on: 08/Feb/23 05:02 Start Date: 08/Feb/23 05:02 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4041: URL: https://github.com/apache/hive/pull/4041#issuecomment-1422018490 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4041) [](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4041=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4041=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4041=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4041=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4041=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4041=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844228) Time Spent: 20m (was: 10m) > Fix improper replication metric count when hive.repl.filter.transactions is > set to true. > > > Key: HIVE-26961 > URL: https://issues.apache.org/jira/browse/HIVE-26961 > Project: Hive > Issue Type: Bug >Reporter: Rakshith C >Assignee: Rakshith C >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Scenario: > when hive.repl.filter.transactions = true, repl dump filters read only > transaction to improve thorughput. > Metrics logged to HMS are improper because there is a mismatch in count > between events read from notification logs and events dumped to staging > directory. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26961) Fix improper replication metric count when hive.repl.filter.transactions is set to true.
[ https://issues.apache.org/jira/browse/HIVE-26961?focusedWorklogId=844224=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844224 ] ASF GitHub Bot logged work on HIVE-26961: - Author: ASF GitHub Bot Created on: 08/Feb/23 04:10 Start Date: 08/Feb/23 04:10 Worklog Time Spent: 10m Work Description: Rakshith606 opened a new pull request, #4041: URL: https://github.com/apache/hive/pull/4041 …ons are filtered. ### What changes were proposed in this pull request? Fixed a bug in Repl Dump to fix dumped events count when open transactions are filtered. ### Why are the changes needed? To ensure appropriate dumped events count are reflected in replication metrics. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Issue Time Tracking --- Worklog Id: (was: 844224) Remaining Estimate: 0h Time Spent: 10m > Fix improper replication metric count when hive.repl.filter.transactions is > set to true. > > > Key: HIVE-26961 > URL: https://issues.apache.org/jira/browse/HIVE-26961 > Project: Hive > Issue Type: Bug >Reporter: Rakshith C >Assignee: Rakshith C >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Scenario: > when hive.repl.filter.transactions = true, repl dump filters read only > transaction to improve thorughput. > Metrics logged to HMS are improper because there is a mismatch in count > between events read from notification logs and events dumped to staging > directory. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-26961) Fix improper replication metric count when hive.repl.filter.transactions is set to true.
[ https://issues.apache.org/jira/browse/HIVE-26961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-26961: -- Labels: pull-request-available (was: ) > Fix improper replication metric count when hive.repl.filter.transactions is > set to true. > > > Key: HIVE-26961 > URL: https://issues.apache.org/jira/browse/HIVE-26961 > Project: Hive > Issue Type: Bug >Reporter: Rakshith C >Assignee: Rakshith C >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Scenario: > when hive.repl.filter.transactions = true, repl dump filters read only > transaction to improve thorughput. > Metrics logged to HMS are improper because there is a mismatch in count > between events read from notification logs and events dumped to staging > directory. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27049) Iceberg: Provide current snapshot version in show-create-table
[ https://issues.apache.org/jira/browse/HIVE-27049?focusedWorklogId=844206=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844206 ] ASF GitHub Bot logged work on HIVE-27049: - Author: ASF GitHub Bot Created on: 08/Feb/23 02:23 Start Date: 08/Feb/23 02:23 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4036: URL: https://github.com/apache/hive/pull/4036#issuecomment-1421886766 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4036) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4036=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4036=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844206) Time Spent: 40m (was: 0.5h) > Iceberg: Provide current snapshot version in show-create-table > -- > > Key: HIVE-27049 > URL: https://issues.apache.org/jira/browse/HIVE-27049 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: Rajesh Balamohan >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > It will be helpful to show "current snapshot" id in "show create table" > statement. This will help in easier debugging. Otherwise, user has to > explicitly query the metadata or read the JSON file to get this info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-25799) Maven should stop early for unsupported java versions
[ https://issues.apache.org/jira/browse/HIVE-25799?focusedWorklogId=844205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844205 ] ASF GitHub Bot logged work on HIVE-25799: - Author: ASF GitHub Bot Created on: 08/Feb/23 00:43 Start Date: 08/Feb/23 00:43 Worklog Time Spent: 10m Work Description: jsoref commented on PR #2866: URL: https://github.com/apache/hive/pull/2866#issuecomment-1421712074 Sigh Issue Time Tracking --- Worklog Id: (was: 844205) Time Spent: 2h 20m (was: 2h 10m) > Maven should stop early for unsupported java versions > - > > Key: HIVE-25799 > URL: https://issues.apache.org/jira/browse/HIVE-25799 > Project: Hive > Issue Type: Improvement > Components: Build Infrastructure >Reporter: Josh Soref >Assignee: Josh Soref >Priority: Trivial > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > I made the mistake of using my default java compiler (which apparently is 17) > to build hive, and it failed. If hive only supports a certain version (or > ranges of versions) of java, it would be incredibly helpful if maven reported > this eagerly. > This is easily achievable using the maven enforcer system (a number of > enforcers are already deployed in the top level pom): > [https://maven.apache.org/enforcer/enforcer-rules/requireJavaVersion.html] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-25799) Maven should stop early for unsupported java versions
[ https://issues.apache.org/jira/browse/HIVE-25799?focusedWorklogId=844202=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844202 ] ASF GitHub Bot logged work on HIVE-25799: - Author: ASF GitHub Bot Created on: 08/Feb/23 00:20 Start Date: 08/Feb/23 00:20 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on PR #2866: URL: https://github.com/apache/hive/pull/2866#issuecomment-1421689231 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. Issue Time Tracking --- Worklog Id: (was: 844202) Time Spent: 2h 10m (was: 2h) > Maven should stop early for unsupported java versions > - > > Key: HIVE-25799 > URL: https://issues.apache.org/jira/browse/HIVE-25799 > Project: Hive > Issue Type: Improvement > Components: Build Infrastructure >Reporter: Josh Soref >Assignee: Josh Soref >Priority: Trivial > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > I made the mistake of using my default java compiler (which apparently is 17) > to build hive, and it failed. If hive only supports a certain version (or > ranges of versions) of java, it would be incredibly helpful if maven reported > this eagerly. > This is easily achievable using the maven enforcer system (a number of > enforcers are already deployed in the top level pom): > [https://maven.apache.org/enforcer/enforcer-rules/requireJavaVersion.html] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27049) Iceberg: Provide current snapshot version in show-create-table
[ https://issues.apache.org/jira/browse/HIVE-27049?focusedWorklogId=844191=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844191 ] ASF GitHub Bot logged work on HIVE-27049: - Author: ASF GitHub Bot Created on: 07/Feb/23 23:12 Start Date: 07/Feb/23 23:12 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4036: URL: https://github.com/apache/hive/pull/4036#issuecomment-1421599871 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4036) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4036=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4036=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844191) Time Spent: 0.5h (was: 20m) > Iceberg: Provide current snapshot version in show-create-table > -- > > Key: HIVE-27049 > URL: https://issues.apache.org/jira/browse/HIVE-27049 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: Rajesh Balamohan >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > It will be helpful to show "current snapshot" id in "show create table" > statement. This will help in easier debugging. Otherwise, user has to > explicitly query the metadata or read the JSON file to get this info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26804) Cancel Compactions in initiated state
[ https://issues.apache.org/jira/browse/HIVE-26804?focusedWorklogId=844175=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844175 ] ASF GitHub Bot logged work on HIVE-26804: - Author: ASF GitHub Bot Created on: 07/Feb/23 20:13 Start Date: 07/Feb/23 20:13 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3880: URL: https://github.com/apache/hive/pull/3880#issuecomment-1421381807 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3880) [](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3880=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3880=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3880=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3880=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3880=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3880=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844175) Time Spent: 5h 40m (was: 5.5h) > Cancel Compactions in initiated state > - > > Key: HIVE-26804 > URL: https://issues.apache.org/jira/browse/HIVE-26804 > Project: Hive > Issue Type: New Feature > Components: Hive >Reporter: KIRTI RUGE >Assignee: KIRTI RUGE >Priority: Major > Labels: pull-request-available > Time Spent: 5h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26954) Upgrade Avro to 1.11.1
[ https://issues.apache.org/jira/browse/HIVE-26954?focusedWorklogId=844170=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844170 ] ASF GitHub Bot logged work on HIVE-26954: - Author: ASF GitHub Bot Created on: 07/Feb/23 19:24 Start Date: 07/Feb/23 19:24 Worklog Time Spent: 10m Work Description: ayushtkn commented on PR #4012: URL: https://github.com/apache/hive/pull/4012#issuecomment-1421325965 I am not sure what blocks us from moving forward, we have already diverged from hadoop, Hadoop is at 1.7.7 https://github.com/apache/hadoop/blob/branch-3.3.1/hadoop-project/pom.xml#L65-L66 And we are at 1.8.2 already. So, if anything has to break due to hadoop it should have been broken by now, but we haven't got hold of anything till now, the upgrade was done on branch-3 and branch-2 as well([HIVE-19662](https://issues.apache.org/jira/browse/HIVE-19662)), so divergence b/w hadoop and hive avro version is already there. [HIVE-24324](https://issues.apache.org/jira/browse/HIVE-24324) was done to pave way for future upgrades. The tests are green. @akshat0395 can you deploy and try some tests on an actual cluster, if things work. If we don't find any conclusive evidence like. "This breaks due to Avro upgrade", I think we can move forward. @zabetak / @abstractdog any thoughts around this? Issue Time Tracking --- Worklog Id: (was: 844170) Time Spent: 1h 10m (was: 1h) > Upgrade Avro to 1.11.1 > -- > > Key: HIVE-26954 > URL: https://issues.apache.org/jira/browse/HIVE-26954 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Upgrade Avro dependencies to 1.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27029) hive query fails with Filesystem closed error
[
https://issues.apache.org/jira/browse/HIVE-27029?focusedWorklogId=844168=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844168
]
ASF GitHub Bot logged work on HIVE-27029:
-
Author: ASF GitHub Bot
Created on: 07/Feb/23 18:55
Start Date: 07/Feb/23 18:55
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4024:
URL: https://github.com/apache/hive/pull/4024#issuecomment-1421289308
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4024)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=BUG)
[1
Bug](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4024=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4024=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4024=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=CODE_SMELL)
[1 Code
Smell](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=4024=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=4024=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 844168)
Time Spent: 40m (was: 0.5h)
> hive query fails with Filesystem closed error
> -
>
> Key: HIVE-27029
> URL: https://issues.apache.org/jira/browse/HIVE-27029
> Project: Hive
> Issue Type: Bug
>Reporter: Mahesh Raju Somalaraju
>Assignee: Mahesh Raju Somalaraju
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This Jira is raised to modify/fix the code which is done in part of
> *HIVE-26352.*
>
> we should remove the finally block as this is causing the filesystem close
> errors.
> {code:java}
> String queueName, String userName) throws IOException, InterruptedException {
> UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
> try {
> ugi.doAs((PrivilegedExceptionAction < Void > )() - > {
> checkQueueAccessInternal(queueName, userName);
> return null;
> } );
> } finally {
> {
> try {
> FileSystem.closeAllForUGI(ugi);
> } catch (IOException exception) {
> LOG.error("Could not clean up file-system handles for UGI: "
> + ugi, exception);
> } }
> }
> } {code}
>
> Caused by: java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:483)
>
[jira] [Work logged] (HIVE-27019) Split Cleaner into separate manageable modular entities
[ https://issues.apache.org/jira/browse/HIVE-27019?focusedWorklogId=844151=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844151 ] ASF GitHub Bot logged work on HIVE-27019: - Author: ASF GitHub Bot Created on: 07/Feb/23 17:15 Start Date: 07/Feb/23 17:15 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4032: URL: https://github.com/apache/hive/pull/4032#issuecomment-1421137452 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4032) [](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4032=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4032=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4032=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=CODE_SMELL) [41 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4032=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4032=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4032=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844151) Time Spent: 20m (was: 10m) > Split Cleaner into separate manageable modular entities > --- > > Key: HIVE-27019 > URL: https://issues.apache.org/jira/browse/HIVE-27019 > Project: Hive > Issue Type: Sub-task >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > As described by the parent task - > Cleaner can be divided into separate entities like - > *1) Handler* - This entity fetches the data from the metastore DB from > relevant tables and converts it into a request entity called CleaningRequest. > It would also do SQL operations post cleanup (postprocess). Every type of > cleaning request is provided by a separate handler. > *2) Filesystem remover* - This entity fetches the cleaning requests from > various handlers and deletes them according to the cleaning request. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27030) Upgrade Hive to Hadoop 3.3.4 (Latest release)
[ https://issues.apache.org/jira/browse/HIVE-27030?focusedWorklogId=844150=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844150 ] ASF GitHub Bot logged work on HIVE-27030: - Author: ASF GitHub Bot Created on: 07/Feb/23 17:12 Start Date: 07/Feb/23 17:12 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4025: URL: https://github.com/apache/hive/pull/4025#issuecomment-1421130592 @ayushtkn You are right when we try to integrate it with Tez, things can break. I agree fully (as you pointed out). **But in that case, we can ignore this commit in Hive 4.x release.** We can still have this PR merged to Hive master I think. The failure you are talking about is because Tez 0.10.2 uses 3.3.1 Hadoop and Hive will use 3.3.4 if this PR goes through. This is because Tez 0.10.2 brings in certain older jars in the classpath while a Hive job is executed as transitive dependencies for Hadoop 3.3.1. This was exactly why I was pushing for Tez 0.10.3 release which will have Hadoop 3.3.4 as we were discussing on this thread - https://github.com/apache/tez/pull/213. Personally, I have added internal commits in Tez 0.10.2 just for testing in my local cluster where I just upgraded Hadoop to 3.3.4 (in Tez 0.10.2) and used that SNAPSHOT jar in the Hive master pom. Those errors were resolved. So basically, what I am again suggesting is that we can have a Tez 0.10.3 release (with Hadoop 3.3.4), if everyone agrees. So that would mean Hive master, Tez 0.10.3 and Hadoop 3.3.4 can form a proper stack for end to end integration. Also, this will help in the ongoing hive-3.2.0 release that we are putting our efforts to where we are trying to use Hadoop 3.3.4 as the latest stack. Please let me know your opinion. cc @zabetak @abstractdog Issue Time Tracking --- Worklog Id: (was: 844150) Time Spent: 40m (was: 0.5h) > Upgrade Hive to Hadoop 3.3.4 (Latest release) > - > > Key: HIVE-27030 > URL: https://issues.apache.org/jira/browse/HIVE-27030 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Upgrading HIVE to latest Hadoop 3.3.4 release -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27039) Backport HIVE-21924 Split text files even if header/footer exists
[ https://issues.apache.org/jira/browse/HIVE-27039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27039: -- Labels: pull-request-available (was: ) > Backport HIVE-21924 Split text files even if header/footer exists > - > > Key: HIVE-27039 > URL: https://issues.apache.org/jira/browse/HIVE-27039 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27039) Backport HIVE-21924 Split text files even if header/footer exists
[ https://issues.apache.org/jira/browse/HIVE-27039?focusedWorklogId=844149=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844149 ] ASF GitHub Bot logged work on HIVE-27039: - Author: ASF GitHub Bot Created on: 07/Feb/23 17:00 Start Date: 07/Feb/23 17:00 Worklog Time Spent: 10m Work Description: Indhumathi27 opened a new pull request, #4040: URL: https://github.com/apache/hive/pull/4040 ### What changes were proposed in this pull request? Backport HIVE-21924: Split text files even if header/footer exists ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? New test cases added Issue Time Tracking --- Worklog Id: (was: 844149) Remaining Estimate: 0h Time Spent: 10m > Backport HIVE-21924 Split text files even if header/footer exists > - > > Key: HIVE-27039 > URL: https://issues.apache.org/jira/browse/HIVE-27039 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27037) Backport HIVE-24378 Leading and trailing spaces are not removed before decimal conversion
[ https://issues.apache.org/jira/browse/HIVE-27037?focusedWorklogId=844146=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844146 ] ASF GitHub Bot logged work on HIVE-27037: - Author: ASF GitHub Bot Created on: 07/Feb/23 16:40 Start Date: 07/Feb/23 16:40 Worklog Time Spent: 10m Work Description: Indhumathi27 opened a new pull request, #4039: URL: https://github.com/apache/hive/pull/4039 ### What changes were proposed in this pull request? Backport HIVE-24378: Leading and trailing spaces are not removed before decimal conversion ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? New test case added Issue Time Tracking --- Worklog Id: (was: 844146) Remaining Estimate: 0h Time Spent: 10m > Backport HIVE-24378 Leading and trailing spaces are not removed before > decimal conversion > - > > Key: HIVE-27037 > URL: https://issues.apache.org/jira/browse/HIVE-27037 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > The decimal conversion is not removing the extra spaces in some scenarios. > because of this the numbers are getting converted to null. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27037) Backport HIVE-24378 Leading and trailing spaces are not removed before decimal conversion
[ https://issues.apache.org/jira/browse/HIVE-27037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27037: -- Labels: pull-request-available (was: ) > Backport HIVE-24378 Leading and trailing spaces are not removed before > decimal conversion > - > > Key: HIVE-27037 > URL: https://issues.apache.org/jira/browse/HIVE-27037 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The decimal conversion is not removing the extra spaces in some scenarios. > because of this the numbers are getting converted to null. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[
https://issues.apache.org/jira/browse/HIVE-27056?focusedWorklogId=844140=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844140
]
ASF GitHub Bot logged work on HIVE-27056:
-
Author: ASF GitHub Bot
Created on: 07/Feb/23 16:08
Start Date: 07/Feb/23 16:08
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4037:
URL: https://github.com/apache/hive/pull/4037#issuecomment-1421026141
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4037)
[](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4037=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4037=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4037=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=CODE_SMELL)
[5 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=4037=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=4037=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=4037=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 844140)
Time Spent: 20m (was: 10m)
> Ensure that MR distcp goes to the same Yarn queue as other parts of the query
> -
>
> Key: HIVE-27056
> URL: https://issues.apache.org/jira/browse/HIVE-27056
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Attachments: image (1).png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Given the following plan for an EXPORT TABLE query:
> {code}
> ++
> | Explain |
> ++
> | STAGE DEPENDENCIES:|
> | Stage-2 is a root stage |
> | Stage-3 depends on stages: Stage-2 |
> | Stage-1 depends on stages: Stage-3 |
> | Stage-6 depends on stages: Stage-1 |
> | Stage-5 depends on stages: Stage-6 |
> | Stage-7 depends on stages: Stage-5 |
> ||
> | STAGE PLANS: |
> | Stage: Stage-2 |
> | Tez|
> | DagId:
[jira] [Work logged] (HIVE-27049) Iceberg: Provide current snapshot version in show-create-table
[ https://issues.apache.org/jira/browse/HIVE-27049?focusedWorklogId=844122=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844122 ] ASF GitHub Bot logged work on HIVE-27049: - Author: ASF GitHub Bot Created on: 07/Feb/23 13:58 Start Date: 07/Feb/23 13:58 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4036: URL: https://github.com/apache/hive/pull/4036#issuecomment-1420819661 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4036) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4036=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [3 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4036=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4036=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4036=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844122) Time Spent: 20m (was: 10m) > Iceberg: Provide current snapshot version in show-create-table > -- > > Key: HIVE-27049 > URL: https://issues.apache.org/jira/browse/HIVE-27049 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: Rajesh Balamohan >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > It will be helpful to show "current snapshot" id in "show create table" > statement. This will help in easier debugging. Otherwise, user has to > explicitly query the metadata or read the JSON file to get this info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27053) Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in COALESCE / ELT
[ https://issues.apache.org/jira/browse/HIVE-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685309#comment-17685309 ] Indhumathi Muthumurugesh commented on HIVE-27053: - [~zabetak] By mistake, updated the wrong version in the jira. Thanks for the update. > Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in > COALESCE / ELT > > > Key: HIVE-27053 > URL: https://issues.apache.org/jira/browse/HIVE-27053 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27053) Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in COALESCE / ELT
[ https://issues.apache.org/jira/browse/HIVE-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27053: Affects Version/s: 3.1.3 > Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in > COALESCE / ELT > > > Key: HIVE-27053 > URL: https://issues.apache.org/jira/browse/HIVE-27053 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27047) Backport HIVE-24514 UpdateMDatabaseURI does not update managed location URI
[ https://issues.apache.org/jira/browse/HIVE-27047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27047: Target Version/s: (was: 3.0.0) Affects Version/s: 3.1.3 > Backport HIVE-24514 UpdateMDatabaseURI does not update managed location URI > --- > > Key: HIVE-27047 > URL: https://issues.apache.org/jira/browse/HIVE-27047 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27046) Backport HIVE-24178 managedlocation is missing in SHOW CREATE DATABASE
[ https://issues.apache.org/jira/browse/HIVE-27046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27046: Target Version/s: (was: 3.0.0) Affects Version/s: 3.1.3 > Backport HIVE-24178 managedlocation is missing in SHOW CREATE DATABASE > -- > > Key: HIVE-27046 > URL: https://issues.apache.org/jira/browse/HIVE-27046 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27044) Backport HIVE-22995 Add support for location for managed tables on database
[ https://issues.apache.org/jira/browse/HIVE-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27044: Target Version/s: (was: 3.0.0) Affects Version/s: 3.1.3 > Backport HIVE-22995 Add support for location for managed tables on database > --- > > Key: HIVE-27044 > URL: https://issues.apache.org/jira/browse/HIVE-27044 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27045) Backport HIVE-23316 Add tests to cover database managed location related DDL and fix minor issues
[ https://issues.apache.org/jira/browse/HIVE-27045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27045: Target Version/s: (was: 3.0.0) Affects Version/s: 3.1.3 > Backport HIVE-23316 Add tests to cover database managed location related DDL > and fix minor issues > - > > Key: HIVE-27045 > URL: https://issues.apache.org/jira/browse/HIVE-27045 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27042) Backport HIVE-24151 MultiDelimitSerDe shifts data if strings contain non-ASCII characters
[ https://issues.apache.org/jira/browse/HIVE-27042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27042: Fix Version/s: (was: 4.0.0-alpha-1) Target Version/s: (was: 3.0.0) Affects Version/s: 3.1.3 > Backport HIVE-24151 MultiDelimitSerDe shifts data if strings contain > non-ASCII characters > - > > Key: HIVE-27042 > URL: https://issues.apache.org/jira/browse/HIVE-27042 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27041) Backport HIVE-22360 MultiDelimitSerDe returns wrong results in last column when the loaded file has more columns than those in table schema
[ https://issues.apache.org/jira/browse/HIVE-27041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27041: Fix Version/s: (was: 4.0.0-alpha-1) Target Version/s: (was: 3.0.0) Affects Version/s: 3.1.3 > Backport HIVE-22360 MultiDelimitSerDe returns wrong results in last column > when the loaded file has more columns than those in table schema > --- > > Key: HIVE-27041 > URL: https://issues.apache.org/jira/browse/HIVE-27041 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27040) Backport HIVE-20619 Include MultiDelimitSerDe in HIveServer2 By Default
[ https://issues.apache.org/jira/browse/HIVE-27040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27040: Fix Version/s: (was: 4.0.0-alpha-1) Target Version/s: (was: 3.0.0) Affects Version/s: 3.1.3 > Backport HIVE-20619 Include MultiDelimitSerDe in HIveServer2 By Default > --- > > Key: HIVE-27040 > URL: https://issues.apache.org/jira/browse/HIVE-27040 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27038) Backport [Partial] HIVE-21838 Allow Truncate external table when 'external.table.purge' property is true"
[ https://issues.apache.org/jira/browse/HIVE-27038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27038: Affects Version/s: 3.1.3 > Backport [Partial] HIVE-21838 Allow Truncate external table when > 'external.table.purge' property is true" > -- > > Key: HIVE-27038 > URL: https://issues.apache.org/jira/browse/HIVE-27038 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27039) Backport HIVE-21924 Split text files even if header/footer exists
[ https://issues.apache.org/jira/browse/HIVE-27039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27039: Fix Version/s: (was: 4.0.0-alpha-1) Target Version/s: (was: 3.0.0) Affects Version/s: 3.1.3 > Backport HIVE-21924 Split text files even if header/footer exists > - > > Key: HIVE-27039 > URL: https://issues.apache.org/jira/browse/HIVE-27039 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27057) Revert "HIVE-19166: TestMiniLlapLocalCliDriver sysdb failure"
[ https://issues.apache.org/jira/browse/HIVE-27057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Raj reassigned HIVE-27057: --- > Revert "HIVE-19166: TestMiniLlapLocalCliDriver sysdb failure" > - > > Key: HIVE-27057 > URL: https://issues.apache.org/jira/browse/HIVE-27057 > Project: Hive > Issue Type: Sub-task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > > The sysdb test fails after this commit was added on branch-3 so reverting it. > Before this commit sysdb works fine on branch-3. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27037) Backport HIVE-24378 Leading and trailing spaces are not removed before decimal conversion
[ https://issues.apache.org/jira/browse/HIVE-27037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27037: Affects Version/s: 3.1.3 > Backport HIVE-24378 Leading and trailing spaces are not removed before > decimal conversion > - > > Key: HIVE-27037 > URL: https://issues.apache.org/jira/browse/HIVE-27037 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.3 >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > > The decimal conversion is not removing the extra spaces in some scenarios. > because of this the numbers are getting converted to null. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27038) Backport [Partial] HIVE-21838 Allow Truncate external table when 'external.table.purge' property is true"
[ https://issues.apache.org/jira/browse/HIVE-27038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27038: Fix Version/s: (was: 4.0.0-alpha-1) Target Version/s: (was: 3.0.0) > Backport [Partial] HIVE-21838 Allow Truncate external table when > 'external.table.purge' property is true" > -- > > Key: HIVE-27038 > URL: https://issues.apache.org/jira/browse/HIVE-27038 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27037) Backport HIVE-24378 Leading and trailing spaces are not removed before decimal conversion
[ https://issues.apache.org/jira/browse/HIVE-27037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27037: Fix Version/s: (was: 4.0.0-alpha-1) Target Version/s: (was: 3.0.0) > Backport HIVE-24378 Leading and trailing spaces are not removed before > decimal conversion > - > > Key: HIVE-27037 > URL: https://issues.apache.org/jira/browse/HIVE-27037 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > > The decimal conversion is not removing the extra spaces in some scenarios. > because of this the numbers are getting converted to null. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27044) Backport HIVE-22995 Add support for location for managed tables on database
[ https://issues.apache.org/jira/browse/HIVE-27044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh updated HIVE-27044: Fix Version/s: (was: 4.0.0-alpha-1) > Backport HIVE-22995 Add support for location for managed tables on database > --- > > Key: HIVE-27044 > URL: https://issues.apache.org/jira/browse/HIVE-27044 > Project: Hive > Issue Type: Improvement >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27055) hive-exec typos part 3
[ https://issues.apache.org/jira/browse/HIVE-27055?focusedWorklogId=844106=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844106 ] ASF GitHub Bot logged work on HIVE-27055: - Author: ASF GitHub Bot Created on: 07/Feb/23 12:25 Start Date: 07/Feb/23 12:25 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #4035: URL: https://github.com/apache/hive/pull/4035#issuecomment-1420689874 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4035) [](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4035=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4035=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4035=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=CODE_SMELL) [7 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=4035=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=4035=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=4035=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 844106) Time Spent: 20m (was: 10m) > hive-exec typos part 3 > -- > > Key: HIVE-27055 > URL: https://issues.apache.org/jira/browse/HIVE-27055 > Project: Hive > Issue Type: Improvement > Components: Query Planning, Query Processor >Affects Versions: 4.0.0-alpha-2 >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Trivial > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > multiple typos and grammar errors in hive-exec module code and comments -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure
[ https://issues.apache.org/jira/browse/HIVE-19166?focusedWorklogId=844104=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844104 ] ASF GitHub Bot logged work on HIVE-19166: - Author: ASF GitHub Bot Created on: 07/Feb/23 12:16 Start Date: 07/Feb/23 12:16 Worklog Time Spent: 10m Work Description: amanraj2520 commented on PR #4018: URL: https://github.com/apache/hive/pull/4018#issuecomment-1420679736 @zabetak @abstractdog This sysdb test was not there before this PR was merged and this test started to fail after this PR was merged. We need to revert this. Issue Time Tracking --- Worklog Id: (was: 844104) Time Spent: 20m (was: 10m) > TestMiniLlapLocalCliDriver sysdb failure > > > Key: HIVE-19166 > URL: https://issues.apache.org/jira/browse/HIVE-19166 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Daniel Dai >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0, 4.0.0-alpha-1 > > Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch, > HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch, > HIVE-19166.10.patch, HIVE-19166.11.patch, HIVE-19166.12.patch, > HIVE-19166.13.patch, HIVE-19166.14.patch, HIVE-19166.15.patch, > HIVE-19166.16.patch, HIVE-19166.17.patch, HIVE-19166.18.patch, > HIVE-19166.19.patch, HIVE-19166.2.patch, HIVE-19166.20.branch-3.patch, > HIVE-19166.20.patch, HIVE-19166.3.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Broken by HIVE-18715 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27025) Test fix for sharedwork.q
[
https://issues.apache.org/jira/browse/HIVE-27025?focusedWorklogId=844102=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844102
]
ASF GitHub Bot logged work on HIVE-27025:
-
Author: ASF GitHub Bot
Created on: 07/Feb/23 12:13
Start Date: 07/Feb/23 12:13
Worklog Time Spent: 10m
Work Description: amanraj2520 commented on PR #4017:
URL: https://github.com/apache/hive/pull/4017#issuecomment-1420676281
@zabetak @abstractdog Can you please review and merge this.
Issue Time Tracking
---
Worklog Id: (was: 844102)
Time Spent: 0.5h (was: 20m)
> Test fix for sharedwork.q
> -
>
> Key: HIVE-27025
> URL: https://issues.apache.org/jira/browse/HIVE-27025
> Project: Hive
> Issue Type: Sub-task
>Reporter: Aman Raj
>Assignee: Aman Raj
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> sharedwork.q fails with the following error because of differences :
> {code:java}
> java.lang.AssertionError: Client Execution succeeded but contained
> differences (error code = 1) after executing sharedwork.q 126c126<
> predicate: ((col_20) IN ('part1', 'part2', 'part3') and CAST( col_3
> AS DATE) BETWEEN DATE'2018-07-01' AND DATE'2019-01-23') (type: boolean)--->
> predicate: ((col_20) IN ('part1', 'part2', 'part3') and
> (col_7 is not null or col_7 is null) and CAST( col_3 AS DATE) BETWEEN
> DATE'2018-07-01' AND DATE'2019-01-23') (type: boolean) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[
https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27056:
Description:
Given the following plan for an EXPORT TABLE query:
{code}
++
| Explain |
++
| STAGE DEPENDENCIES:|
| Stage-2 is a root stage |
| Stage-3 depends on stages: Stage-2 |
| Stage-1 depends on stages: Stage-3 |
| Stage-6 depends on stages: Stage-1 |
| Stage-5 depends on stages: Stage-6 |
| Stage-7 depends on stages: Stage-5 |
||
| STAGE PLANS: |
| Stage: Stage-2 |
| Tez|
| DagId: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
| DagName: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
| Vertices:|
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: distcptest|
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| Select Operator |
| expressions: id (type: int), txt (type: string) |
| outputColumnNames: _col0, _col1 |
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| File Output Operator |
| compressed: false|
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| table: |
| input format:
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
| output format:
org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
| serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
| name:
bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
| Execution mode: vectorized |
||
| Stage: Stage-3 |
| Dependency Collection |
||
| Stage: Stage-1 |
| Move Operator |
| tables: |
| replace: false |
| table: |
| input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
| output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
|
| serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
| name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523
|
||
| Stage: Stage-6 |
| Set Properties |
| table name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
| properties: |
| transactional true |
||
| Stage: Stage-5 |
| Export Work|
||
| Stage: Stage-7 |
| Drop Table |
| table: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
||
++
{code}
here, Stage-5 Export Work starts MR distcp jobs
the problem and solution are described in code comment, pasted below:
{code}
/**
* This method ensures if there is an explicit tez.queue.name set, the hadoop
shim will submit jobs
* to the same yarn queue. This solves a security issue where e.g settings
have the following values:
* tez.queue.name=sample
* hive.server2.tez.queue.access.check=true
* In this case, when a query submits Tez DAGs, the tez client layer checks
whether the end user has access to
* the yarn queue 'sample' via YarnQueueHelper, but this is not respected in
case of MR jobs that run
* even if the query execution engine is Tez. E.g. an EXPORT TABLE can submit
DistCp MR jobs at some
[jira] [Updated] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[
https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27056:
Description:
Given the following plan for an EXPORT TABLE query:
{code}
++
| Explain |
++
| STAGE DEPENDENCIES:|
| Stage-2 is a root stage |
| Stage-3 depends on stages: Stage-2 |
| Stage-1 depends on stages: Stage-3 |
| Stage-6 depends on stages: Stage-1 |
| Stage-5 depends on stages: Stage-6 |
| Stage-7 depends on stages: Stage-5 |
||
| STAGE PLANS: |
| Stage: Stage-2 |
| Tez|
| DagId: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
| DagName: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
| Vertices:|
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: distcptest|
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| Select Operator |
| expressions: id (type: int), txt (type: string) |
| outputColumnNames: _col0, _col1 |
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| File Output Operator |
| compressed: false|
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| table: |
| input format:
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
| output format:
org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
| serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
| name:
bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
| Execution mode: vectorized |
||
| Stage: Stage-3 |
| Dependency Collection |
||
| Stage: Stage-1 |
| Move Operator |
| tables: |
| replace: false |
| table: |
| input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
| output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
|
| serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
| name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523
|
||
| Stage: Stage-6 |
| Set Properties |
| table name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
| properties: |
| transactional true |
||
| Stage: Stage-5 |
| Export Work|
||
| Stage: Stage-7 |
| Drop Table |
| table: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
||
++
{code}
here, Stage-5 Export Work starts MR distcp jobs
the problem and solution is described in code comment
{code}
/**
* This method ensures if there is an explicit tez.queue.name set, the hadoop
shim will submit jobs
* to the same yarn queue. This solves a security issue where e.g settings
have the following values:
* tez.queue.name=sample
* hive.server2.tez.queue.access.check=true
* In this case, when a query submits Tez DAGs, the tez client layer checks
whether the end user has access to
* the yarn queue 'sample' via YarnQueueHelper, but this is not respected in
case of MR jobs that run
* even if the query execution engine is Tez. E.g. an EXPORT TABLE can submit
DistCp MR jobs at some stages when
*
[jira] [Updated] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[
https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27056:
Attachment: image (1).png
> Ensure that MR distcp goes to the same Yarn queue as other parts of the query
> -
>
> Key: HIVE-27056
> URL: https://issues.apache.org/jira/browse/HIVE-27056
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Attachments: image (1).png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Given the following plan for an EXPORT TABLE query:
> {code}
> ++
> | Explain |
> ++
> | STAGE DEPENDENCIES:|
> | Stage-2 is a root stage |
> | Stage-3 depends on stages: Stage-2 |
> | Stage-1 depends on stages: Stage-3 |
> | Stage-6 depends on stages: Stage-1 |
> | Stage-5 depends on stages: Stage-6 |
> | Stage-7 depends on stages: Stage-5 |
> ||
> | STAGE PLANS: |
> | Stage: Stage-2 |
> | Tez|
> | DagId: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
> | DagName: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
> | Vertices:|
> | Map 1 |
> | Map Operator Tree: |
> | TableScan |
> | alias: distcptest|
> | Statistics: Num rows: 2 Data size: 182658 Basic stats:
> COMPLETE Column stats: COMPLETE |
> | Select Operator |
> | expressions: id (type: int), txt (type: string) |
> | outputColumnNames: _col0, _col1 |
> | Statistics: Num rows: 2 Data size: 182658 Basic stats:
> COMPLETE Column stats: COMPLETE |
> | File Output Operator |
> | compressed: false|
> | Statistics: Num rows: 2 Data size: 182658 Basic
> stats: COMPLETE Column stats: COMPLETE |
> | table: |
> | input format:
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
> | output format:
> org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
> | serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
> | name:
> bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
> | Execution mode: vectorized |
> ||
> | Stage: Stage-3 |
> | Dependency Collection |
> ||
> | Stage: Stage-1 |
> | Move Operator |
> | tables: |
> | replace: false |
> | table: |
> | input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
> |
> | output format:
> org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
> | serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
> | name:
> bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
> ||
> | Stage: Stage-6 |
> | Set Properties |
> | table name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523
> |
> | properties: |
> | transactional true |
> ||
> | Stage: Stage-5 |
> | Export Work|
> ||
> | Stage: Stage-7 |
> | Drop Table |
> | table: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
> ||
>
[jira] [Updated] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[
https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27056:
--
Labels: pull-request-available (was: )
> Ensure that MR distcp goes to the same Yarn queue as other parts of the query
> -
>
> Key: HIVE-27056
> URL: https://issues.apache.org/jira/browse/HIVE-27056
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Given the following plan for an EXPORT TABLE query:
> {code}
> ++
> | Explain |
> ++
> | STAGE DEPENDENCIES:|
> | Stage-2 is a root stage |
> | Stage-3 depends on stages: Stage-2 |
> | Stage-1 depends on stages: Stage-3 |
> | Stage-6 depends on stages: Stage-1 |
> | Stage-5 depends on stages: Stage-6 |
> | Stage-7 depends on stages: Stage-5 |
> ||
> | STAGE PLANS: |
> | Stage: Stage-2 |
> | Tez|
> | DagId: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
> | DagName: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
> | Vertices:|
> | Map 1 |
> | Map Operator Tree: |
> | TableScan |
> | alias: distcptest|
> | Statistics: Num rows: 2 Data size: 182658 Basic stats:
> COMPLETE Column stats: COMPLETE |
> | Select Operator |
> | expressions: id (type: int), txt (type: string) |
> | outputColumnNames: _col0, _col1 |
> | Statistics: Num rows: 2 Data size: 182658 Basic stats:
> COMPLETE Column stats: COMPLETE |
> | File Output Operator |
> | compressed: false|
> | Statistics: Num rows: 2 Data size: 182658 Basic
> stats: COMPLETE Column stats: COMPLETE |
> | table: |
> | input format:
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
> | output format:
> org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
> | serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
> | name:
> bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
> | Execution mode: vectorized |
> ||
> | Stage: Stage-3 |
> | Dependency Collection |
> ||
> | Stage: Stage-1 |
> | Move Operator |
> | tables: |
> | replace: false |
> | table: |
> | input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
> |
> | output format:
> org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
> | serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
> | name:
> bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
> ||
> | Stage: Stage-6 |
> | Set Properties |
> | table name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523
> |
> | properties: |
> | transactional true |
> ||
> | Stage: Stage-5 |
> | Export Work|
> ||
> | Stage: Stage-7 |
> | Drop Table |
> | table: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
> ||
>
[jira] [Work logged] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[
https://issues.apache.org/jira/browse/HIVE-27056?focusedWorklogId=844098=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844098
]
ASF GitHub Bot logged work on HIVE-27056:
-
Author: ASF GitHub Bot
Created on: 07/Feb/23 11:50
Start Date: 07/Feb/23 11:50
Worklog Time Spent: 10m
Work Description: abstractdog opened a new pull request, #4037:
URL: https://github.com/apache/hive/pull/4037
…r parts of the query
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
Issue Time Tracking
---
Worklog Id: (was: 844098)
Remaining Estimate: 0h
Time Spent: 10m
> Ensure that MR distcp goes to the same Yarn queue as other parts of the query
> -
>
> Key: HIVE-27056
> URL: https://issues.apache.org/jira/browse/HIVE-27056
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0-alpha-2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Given the following plan for an EXPORT TABLE query:
> {code}
> ++
> | Explain |
> ++
> | STAGE DEPENDENCIES:|
> | Stage-2 is a root stage |
> | Stage-3 depends on stages: Stage-2 |
> | Stage-1 depends on stages: Stage-3 |
> | Stage-6 depends on stages: Stage-1 |
> | Stage-5 depends on stages: Stage-6 |
> | Stage-7 depends on stages: Stage-5 |
> ||
> | STAGE PLANS: |
> | Stage: Stage-2 |
> | Tez|
> | DagId: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
> | DagName: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
> | Vertices:|
> | Map 1 |
> | Map Operator Tree: |
> | TableScan |
> | alias: distcptest|
> | Statistics: Num rows: 2 Data size: 182658 Basic stats:
> COMPLETE Column stats: COMPLETE |
> | Select Operator |
> | expressions: id (type: int), txt (type: string) |
> | outputColumnNames: _col0, _col1 |
> | Statistics: Num rows: 2 Data size: 182658 Basic stats:
> COMPLETE Column stats: COMPLETE |
> | File Output Operator |
> | compressed: false|
> | Statistics: Num rows: 2 Data size: 182658 Basic
> stats: COMPLETE Column stats: COMPLETE |
> | table: |
> | input format:
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
> | output format:
> org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
> | serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
> | name:
> bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
> | Execution mode: vectorized |
> ||
> | Stage: Stage-3 |
> | Dependency Collection |
> ||
> | Stage: Stage-1 |
> | Move Operator |
> | tables: |
> | replace: false |
> | table: |
> | input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
> |
> | output format:
> org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
> | serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
> | name:
> bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
> ||
> | Stage: Stage-6 |
> | Set Properties |
> | table name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523
> |
> |
[jira] [Updated] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[
https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27056:
Description:
Given the following plan for an EXPORT TABLE query:
{code}
++
| Explain |
++
| STAGE DEPENDENCIES:|
| Stage-2 is a root stage |
| Stage-3 depends on stages: Stage-2 |
| Stage-1 depends on stages: Stage-3 |
| Stage-6 depends on stages: Stage-1 |
| Stage-5 depends on stages: Stage-6 |
| Stage-7 depends on stages: Stage-5 |
||
| STAGE PLANS: |
| Stage: Stage-2 |
| Tez|
| DagId: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
| DagName: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
| Vertices:|
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: distcptest|
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| Select Operator |
| expressions: id (type: int), txt (type: string) |
| outputColumnNames: _col0, _col1 |
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| File Output Operator |
| compressed: false|
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| table: |
| input format:
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
| output format:
org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
| serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
| name:
bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
| Execution mode: vectorized |
||
| Stage: Stage-3 |
| Dependency Collection |
||
| Stage: Stage-1 |
| Move Operator |
| tables: |
| replace: false |
| table: |
| input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
| output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
|
| serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
| name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523
|
||
| Stage: Stage-6 |
| Set Properties |
| table name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
| properties: |
| transactional true |
||
| Stage: Stage-5 |
| Export Work|
||
| Stage: Stage-7 |
| Drop Table |
| table: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
||
++
{code}
here, Stage-5 Export Work starts MR distcp jobs
the problem and solution is described in code comment
{code}
/**
* This method ensures if there is an explicit tez.queue.name set, the hadoop
shim will submit jobs
* to the same yarn queue. This solves a security issue where e.g settings
have the following values:
* tez.queue.name=sample
* hive.server2.tez.queue.access.check=true
* In this case, when a query submits Tez DAGs, the tez client layer checks
whether the end user has access to
* the yarn queue 'sample' via YarnQueueHelper, but this is not respected in
case of MR jobs that run
* even if the query execution engine is Tez. E.g. an EXPORT TABLE can submit
DistCp MR jobs at some stages when
*
[jira] [Updated] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[
https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27056:
Description:
Given the following plan for an EXPORT TABLE query:
{code}
++
| Explain |
++
| STAGE DEPENDENCIES:|
| Stage-2 is a root stage |
| Stage-3 depends on stages: Stage-2 |
| Stage-1 depends on stages: Stage-3 |
| Stage-6 depends on stages: Stage-1 |
| Stage-5 depends on stages: Stage-6 |
| Stage-7 depends on stages: Stage-5 |
||
| STAGE PLANS: |
| Stage: Stage-2 |
| Tez|
| DagId: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
| DagName: hive_20230124103803_b865e63c-4565-4b95-8664-8c983938697a:7 |
| Vertices:|
| Map 1 |
| Map Operator Tree: |
| TableScan |
| alias: distcptest|
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| Select Operator |
| expressions: id (type: int), txt (type: string) |
| outputColumnNames: _col0, _col1 |
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| File Output Operator |
| compressed: false|
| Statistics: Num rows: 2 Data size: 182658 Basic stats:
COMPLETE Column stats: COMPLETE |
| table: |
| input format:
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
| output format:
org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat |
| serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
| name:
bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
| Execution mode: vectorized |
||
| Stage: Stage-3 |
| Dependency Collection |
||
| Stage: Stage-1 |
| Move Operator |
| tables: |
| replace: false |
| table: |
| input format: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat |
| output format: org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat
|
| serde: org.apache.hadoop.hive.ql.io.orc.OrcSerde |
| name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523
|
||
| Stage: Stage-6 |
| Set Properties |
| table name: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
| properties: |
| transactional true |
||
| Stage: Stage-5 |
| Export Work|
||
| Stage: Stage-7 |
| Drop Table |
| table: bigd35368.distcptest_ced9015c_852b_49a4_8c28_4d284d598523 |
||
++
{code}
here, Stage-5 Export Work starts MR distcp jobs
the problem and solution is described in code comment
{code}
/**
* This method ensures if there is an explicit tez.queue.name set, the hadoop
shim will submit jobs
* to the same yarn queue. This solves a security issue where e.g settings
have the following values:
* tez.queue.name=sample
* hive.server2.tez.queue.access.check=true
* In this case, when a query submits Tez DAGs, the tez client layer checks
whether the end user has access to
* the yarn queue 'sample' via YarnQueueHelper, but this is not respected in
case of MR jobs that run
* even if the query execution engine is Tez. E.g. an EXPORT TABLE can submit
DistCp MR jobs at some stages when
*
[jira] [Work logged] (HIVE-26887) Make sure dirPath has the correct permissions
[
https://issues.apache.org/jira/browse/HIVE-26887?focusedWorklogId=844096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844096
]
ASF GitHub Bot logged work on HIVE-26887:
-
Author: ASF GitHub Bot
Created on: 07/Feb/23 11:44
Start Date: 07/Feb/23 11:44
Worklog Time Spent: 10m
Work Description: zabetak commented on PR #3894:
URL: https://github.com/apache/hive/pull/3894#issuecomment-1420640941
@skysiders Apologies for not following up on this but I am still pretty
busy with various things.
> Regarding the second point "programming pattern" you mentioned, in fact,
it is also possible to use Hadoop's underlying FileSystem.create(fs, path,
perm) here.
If this pattern is preferred then why not adopting it here? I saw your
comments above about a potential perf benefit. If that is true then it would be
better to propose this perf improvement in the "Hadoop HDFS" project. It would
be better to have this `if` block (or whatever) in one place rather in many
different places (5 times X 5 projects).
Other than that I don't really like the fact that `FileSystem.create` and
`fs.create` have different behavior; this will always be a problem for
developers when trying to pick which API to use. Again this is not something to
handle here but maybe discuss with the HDFS folks.
Issue Time Tracking
---
Worklog Id: (was: 844096)
Time Spent: 2h (was: 1h 50m)
> Make sure dirPath has the correct permissions
> -
>
> Key: HIVE-26887
> URL: https://issues.apache.org/jira/browse/HIVE-26887
> Project: Hive
> Issue Type: Improvement
>Reporter: Zhang Dongsheng
>Priority: Major
> Labels: backward-incompatible, pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> In the QueryResultsCache function of class QueryResultsCache, there is the
> following code segment
> {code:java}
> private QueryResultsCache(HiveConf configuration) throws IOException {
> ..
> FileSystem fs = cacheDirPath.getFileSystem(conf);
> FsPermission fsPermission = new FsPermission("700");
> fs.mkdirs(cacheDirPath, fsPermission);
> ..
> }
> {code}
> It can be seen that the function will use the mkdirs to create cacheDirPath,
> and the parameters passed in include the path variable cacheDirPath and a
> permission 700. But we haven't confirmed whether the permission is correctly
> assigned to the file.
> The above question is raised because there are two mkdir functions of hadoop,
> {code:java}
> mkdirs(Path f, FsPermission permission)
> {code}
> and
> {code:java}
> mkdirs(FileSystem fs, Path dir, FsPermission permission)
> {code}
> and the first one is used here. The permissions of this function will be
> affected by the underlying umask. Although 700 here will hardly be affected
> by umask, but I think from a rigorous point of view, we should have one more
> permission check and permission grant here.
> And I find same issue in other three methods here.
> In class Context
> {code:java}
> private Path getScratchDir(String scheme, String authority,
> boolean mkdir, String scratchDir) {
> ..
> FileSystem fs = dirPath.getFileSystem(conf);
> dirPath = new Path(fs.makeQualified(dirPath).toString());
> FsPermission fsPermission = new FsPermission(scratchDirPermission);
> if (!fs.mkdirs(dirPath, fsPermission)) {
> throw new RuntimeException("Cannot make directory: "
> + dirPath.toString());
> ..
> }
> {code}
> In class SessionState
> {code:java}
> static void createPath(HiveConf conf, Path path, String permission, boolean
> isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> ..
> if (!fs.mkdirs(path, fsPermission)) {
> throw new IOException("Failed to create directory " + path + " on fs "
> + fs.getUri());
> }
> ..
> }
> {code}
> and in class TezSessionState
> {code:java}
> private Path createTezDir(String sessionId, String suffix) throws IOException
> {
> ..
> Path tezDir = new Path(hdfsScratchDir, TEZ_DIR);
> FileSystem fs = tezDir.getFileSystem(conf);
> FsPermission fsPermission = new FsPermission(HiveConf.getVar(conf,
> HiveConf.ConfVars.SCRATCHDIRPERMISSION));
> fs.mkdirs(tezDir, fsPermission);
> ..
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[ https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27056: Fix Version/s: 4.0.0-alpha-2 > Ensure that MR distcp goes to the same Yarn queue as other parts of the query > - > > Key: HIVE-27056 > URL: https://issues.apache.org/jira/browse/HIVE-27056 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Fix For: 4.0.0-alpha-2 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[ https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-27056 started by László Bodor. --- > Ensure that MR distcp goes to the same Yarn queue as other parts of the query > - > > Key: HIVE-27056 > URL: https://issues.apache.org/jira/browse/HIVE-27056 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27056) Ensure that MR distcp goes to the same Yarn queue as other parts of the query
[ https://issues.apache.org/jira/browse/HIVE-27056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-27056: --- Assignee: László Bodor > Ensure that MR distcp goes to the same Yarn queue as other parts of the query > - > > Key: HIVE-27056 > URL: https://issues.apache.org/jira/browse/HIVE-27056 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26954) Upgrade Avro to 1.11.1
[ https://issues.apache.org/jira/browse/HIVE-26954?focusedWorklogId=844090=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844090 ] ASF GitHub Bot logged work on HIVE-26954: - Author: ASF GitHub Bot Created on: 07/Feb/23 11:25 Start Date: 07/Feb/23 11:25 Worklog Time Spent: 10m Work Description: akshat0395 commented on PR #4012: URL: https://github.com/apache/hive/pull/4012#issuecomment-1420618093 ++ @ayushtkn Would be really helpful if you can share some insights here. Thanks Issue Time Tracking --- Worklog Id: (was: 844090) Time Spent: 1h (was: 50m) > Upgrade Avro to 1.11.1 > -- > > Key: HIVE-26954 > URL: https://issues.apache.org/jira/browse/HIVE-26954 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Akshat Mathur >Assignee: Akshat Mathur >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Upgrade Avro dependencies to 1.11.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27054) Backport HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete events
[ https://issues.apache.org/jira/browse/HIVE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-27054: --- Fix Version/s: (was: 4.0.0-alpha-1) > Backport HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete > events > > > Key: HIVE-27054 > URL: https://issues.apache.org/jira/browse/HIVE-27054 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27054) Backport HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete events
[ https://issues.apache.org/jira/browse/HIVE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685222#comment-17685222 ] Stamatis Zampetakis commented on HIVE-27054: [~Indhumathi27] I removed Fix and Target versions since both 4.0.0-alpha-1 and 3.0.0 are released so it is not clear what the intention is. Please have a look at https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-Guidelines on how to use respective fields. > Backport HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete > events > > > Key: HIVE-27054 > URL: https://issues.apache.org/jira/browse/HIVE-27054 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27054) Backport HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete events
[ https://issues.apache.org/jira/browse/HIVE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-27054: --- Target Version/s: (was: 3.0.0) > Backport HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete > events > > > Key: HIVE-27054 > URL: https://issues.apache.org/jira/browse/HIVE-27054 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27053) Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in COALESCE / ELT
[ https://issues.apache.org/jira/browse/HIVE-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685221#comment-17685221 ] Stamatis Zampetakis commented on HIVE-27053: [~Indhumathi27] I removed Fix and Target versions since both 4.0.0-alpha-1 and 3.0.0 are released so it is not clear what the intention is. Please have a look at https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-Guidelines on how to use respective fields. > Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in > COALESCE / ELT > > > Key: HIVE-27053 > URL: https://issues.apache.org/jira/browse/HIVE-27053 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27053) Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in COALESCE / ELT
[ https://issues.apache.org/jira/browse/HIVE-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-27053: --- Target Version/s: (was: 3.0.0) > Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in > COALESCE / ELT > > > Key: HIVE-27053 > URL: https://issues.apache.org/jira/browse/HIVE-27053 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27053) Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in COALESCE / ELT
[ https://issues.apache.org/jira/browse/HIVE-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-27053: --- Fix Version/s: (was: 4.0.0-alpha-1) > Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in > COALESCE / ELT > > > Key: HIVE-27053 > URL: https://issues.apache.org/jira/browse/HIVE-27053 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27049) Iceberg: Provide current snapshot version in show-create-table
[ https://issues.apache.org/jira/browse/HIVE-27049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27049: -- Labels: pull-request-available (was: ) > Iceberg: Provide current snapshot version in show-create-table > -- > > Key: HIVE-27049 > URL: https://issues.apache.org/jira/browse/HIVE-27049 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: Rajesh Balamohan >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > It will be helpful to show "current snapshot" id in "show create table" > statement. This will help in easier debugging. Otherwise, user has to > explicitly query the metadata or read the JSON file to get this info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27049) Iceberg: Provide current snapshot version in show-create-table
[ https://issues.apache.org/jira/browse/HIVE-27049?focusedWorklogId=844079=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844079 ] ASF GitHub Bot logged work on HIVE-27049: - Author: ASF GitHub Bot Created on: 07/Feb/23 10:39 Start Date: 07/Feb/23 10:39 Worklog Time Spent: 10m Work Description: ayushtkn opened a new pull request, #4036: URL: https://github.com/apache/hive/pull/4036 ### What changes were proposed in this pull request? Add current snapshot id in show create table output for iceberg tables. ### Why are the changes needed? Better Usability ### Does this PR introduce _any_ user-facing change? Yes, Adds current snapshot id in show create table output for iceberg tables. ### How was this patch tested? UT Issue Time Tracking --- Worklog Id: (was: 844079) Remaining Estimate: 0h Time Spent: 10m > Iceberg: Provide current snapshot version in show-create-table > -- > > Key: HIVE-27049 > URL: https://issues.apache.org/jira/browse/HIVE-27049 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: Rajesh Balamohan >Assignee: Ayush Saxena >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > It will be helpful to show "current snapshot" id in "show create table" > statement. This will help in easier debugging. Otherwise, user has to > explicitly query the metadata or read the JSON file to get this info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-26825) Compactor: Cleaner shouldn't fetch table details again and again for partitioned tables
[ https://issues.apache.org/jira/browse/HIVE-26825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Végh resolved HIVE-26825. Fix Version/s: 4.0.0 Resolution: Fixed Merged to master, thanks [~rkirtir] for the patch! > Compactor: Cleaner shouldn't fetch table details again and again for > partitioned tables > --- > > Key: HIVE-26825 > URL: https://issues.apache.org/jira/browse/HIVE-26825 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: KIRTI RUGE >Assignee: KIRTI RUGE >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > Cleaner shouldn't be fetch table/partition details for all its partitions. > When there are large number of databases/tables, it takes lot of time for > Initiator to complete its initial iteration and load on DB also goes higher. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26825) Compactor: Cleaner shouldn't fetch table details again and again for partitioned tables
[ https://issues.apache.org/jira/browse/HIVE-26825?focusedWorklogId=844064=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844064 ] ASF GitHub Bot logged work on HIVE-26825: - Author: ASF GitHub Bot Created on: 07/Feb/23 10:10 Start Date: 07/Feb/23 10:10 Worklog Time Spent: 10m Work Description: veghlaci05 merged PR #3864: URL: https://github.com/apache/hive/pull/3864 Issue Time Tracking --- Worklog Id: (was: 844064) Time Spent: 2h 50m (was: 2h 40m) > Compactor: Cleaner shouldn't fetch table details again and again for > partitioned tables > --- > > Key: HIVE-26825 > URL: https://issues.apache.org/jira/browse/HIVE-26825 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: KIRTI RUGE >Assignee: KIRTI RUGE >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > Cleaner shouldn't be fetch table/partition details for all its partitions. > When there are large number of databases/tables, it takes lot of time for > Initiator to complete its initial iteration and load on DB also goes higher. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27055) hive-exec typos part 3
[ https://issues.apache.org/jira/browse/HIVE-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685184#comment-17685184 ] Michal Lorek commented on HIVE-27055: - sure, my bad > hive-exec typos part 3 > -- > > Key: HIVE-27055 > URL: https://issues.apache.org/jira/browse/HIVE-27055 > Project: Hive > Issue Type: Improvement > Components: Query Planning, Query Processor >Affects Versions: 4.0.0-alpha-2 >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > multiple typos and grammar errors in hive-exec module code and comments -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27055) hive-exec typos part 3
[ https://issues.apache.org/jira/browse/HIVE-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685182#comment-17685182 ] Alessandro Solimando commented on HIVE-27055: - I have changed the "Affect Version/s" field value to 4.0.0-alpha-2 because 4.0.0 is not out yet AFAIK. > hive-exec typos part 3 > -- > > Key: HIVE-27055 > URL: https://issues.apache.org/jira/browse/HIVE-27055 > Project: Hive > Issue Type: Improvement > Components: Query Planning, Query Processor >Affects Versions: 4.0.0-alpha-2 >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > multiple typos and grammar errors in hive-exec module code and comments -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27055) hive-exec typos part 3
[ https://issues.apache.org/jira/browse/HIVE-27055?focusedWorklogId=844056=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844056 ] ASF GitHub Bot logged work on HIVE-27055: - Author: ASF GitHub Bot Created on: 07/Feb/23 09:46 Start Date: 07/Feb/23 09:46 Worklog Time Spent: 10m Work Description: mlorek opened a new pull request, #4035: URL: https://github.com/apache/hive/pull/4035 ### What changes were proposed in this pull request? fix typos in comments and code ### Why are the changes needed? to improve code quality ### Does this PR introduce _any_ user-facing change? No (except debug logging) ### How was this patch tested? build Issue Time Tracking --- Worklog Id: (was: 844056) Remaining Estimate: 0h Time Spent: 10m > hive-exec typos part 3 > -- > > Key: HIVE-27055 > URL: https://issues.apache.org/jira/browse/HIVE-27055 > Project: Hive > Issue Type: Improvement > Components: Query Planning, Query Processor >Affects Versions: 4.0.0-alpha-2 >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > multiple typos and grammar errors in hive-exec module code and comments -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27055) hive-exec typos part 3
[ https://issues.apache.org/jira/browse/HIVE-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27055: -- Labels: pull-request-available (was: ) > hive-exec typos part 3 > -- > > Key: HIVE-27055 > URL: https://issues.apache.org/jira/browse/HIVE-27055 > Project: Hive > Issue Type: Improvement > Components: Query Planning, Query Processor >Affects Versions: 4.0.0-alpha-2 >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > multiple typos and grammar errors in hive-exec module code and comments -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27055) hive-exec typos part 3
[ https://issues.apache.org/jira/browse/HIVE-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Solimando updated HIVE-27055: Affects Version/s: 4.0.0-alpha-2 (was: 4.0.0) > hive-exec typos part 3 > -- > > Key: HIVE-27055 > URL: https://issues.apache.org/jira/browse/HIVE-27055 > Project: Hive > Issue Type: Improvement > Components: Query Planning, Query Processor >Affects Versions: 4.0.0-alpha-2 >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Trivial > > multiple typos and grammar errors in hive-exec module code and comments -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27029) hive query fails with Filesystem closed error
[
https://issues.apache.org/jira/browse/HIVE-27029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27029:
Description:
This Jira is raised to modify/fix the code which is done in part of
*HIVE-26352.*
we should remove the finally block as this is causing the filesystem close
errors.
{code:java}
String queueName, String userName) throws IOException, InterruptedException {
UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
try {
ugi.doAs((PrivilegedExceptionAction < Void > )() - > {
checkQueueAccessInternal(queueName, userName);
return null;
} );
} finally {
{
try {
FileSystem.closeAllForUGI(ugi);
} catch (IOException exception) {
LOG.error("Could not clean up file-system handles for UGI: " +
ugi, exception);
} }
}
} {code}
Caused by: java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:483)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at org.apache.hadoop.hdfs.DFSClient.getEZForPath(DFSClient.java:2771)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem$54.doCall(DistributedFileSystem.java:2796)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem$54.doCall(DistributedFileSystem.java:2793)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
~[hadoop-common-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem.getEZForPath(DistributedFileSystem.java:2812)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.client.HdfsAdmin.getEncryptionZoneForPath(HdfsAdmin.java:374)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.getEncryptionZoneForPath(Hadoop23Shims.java:1384)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
at
org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.isPathEncrypted(Hadoop23Shims.java:1379)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.isPathEncrypted(SemanticAnalyzer.java:2484)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
was:
This Jira is raised to modify/fix the code which is done in part of
*HIVE-26352.*
we should remove the finally block as this is causing the filesystem close
errors.
{code}
String queueName, String userName) throws IOException, InterruptedException {
UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
try {
ugi.doAs((PrivilegedExceptionAction) () ->
{ checkQueueAccessInternal(queueName, userName); return null; }
);
} {color:#0747a6}*finally {*{color}
{color:#0747a6} *try {*{color}
{color:#0747a6} *FileSystem.closeAllForUGI(ugi);*{color}
} catch (IOException exception)
{ LOG.error("Could not clean up file-system handles for UGI: " + ugi,
exception); }
}
}
{code}
Caused by: java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:483)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at org.apache.hadoop.hdfs.DFSClient.getEZForPath(DFSClient.java:2771)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem$54.doCall(DistributedFileSystem.java:2796)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem$54.doCall(DistributedFileSystem.java:2793)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
~[hadoop-common-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem.getEZForPath(DistributedFileSystem.java:2812)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.client.HdfsAdmin.getEncryptionZoneForPath(HdfsAdmin.java:374)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.getEncryptionZoneForPath(Hadoop23Shims.java:1384)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
at
org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.isPathEncrypted(Hadoop23Shims.java:1379)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.isPathEncrypted(SemanticAnalyzer.java:2484)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
> hive query fails with Filesystem closed error
> -
>
> Key: HIVE-27029
> URL: https://issues.apache.org/jira/browse/HIVE-27029
> Project: Hive
> Issue Type: Bug
>Reporter: Mahesh Raju Somalaraju
>Assignee: Mahesh Raju
[jira] [Updated] (HIVE-27029) hive query fails with Filesystem closed error
[
https://issues.apache.org/jira/browse/HIVE-27029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27029:
Description:
This Jira is raised to modify/fix the code which is done in part of
*HIVE-26352.*
we should remove the finally block as this is causing the filesystem close
errors.
{code}
String queueName, String userName) throws IOException, InterruptedException {
UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
try {
ugi.doAs((PrivilegedExceptionAction) () ->
{ checkQueueAccessInternal(queueName, userName); return null; }
);
} {color:#0747a6}*finally {*{color}
{color:#0747a6} *try {*{color}
{color:#0747a6} *FileSystem.closeAllForUGI(ugi);*{color}
} catch (IOException exception)
{ LOG.error("Could not clean up file-system handles for UGI: " + ugi,
exception); }
}
}
{code}
Caused by: java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:483)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at org.apache.hadoop.hdfs.DFSClient.getEZForPath(DFSClient.java:2771)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem$54.doCall(DistributedFileSystem.java:2796)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem$54.doCall(DistributedFileSystem.java:2793)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
~[hadoop-common-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem.getEZForPath(DistributedFileSystem.java:2812)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.client.HdfsAdmin.getEncryptionZoneForPath(HdfsAdmin.java:374)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.getEncryptionZoneForPath(Hadoop23Shims.java:1384)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
at
org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.isPathEncrypted(Hadoop23Shims.java:1379)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.isPathEncrypted(SemanticAnalyzer.java:2484)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
was:
This Jira is raised to modify/fix the code which is done in part of
*HIVE-26352.*
we should remove the finally block as this is causing the filesystem close
errors.
String queueName, String userName) throws IOException, InterruptedException {
UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
try {
ugi.doAs((PrivilegedExceptionAction) () -> {
checkQueueAccessInternal(queueName, userName);
return null;
});
} {color:#0747a6}*finally {*{color}
{color:#0747a6} *try {*{color}
{color:#0747a6} *FileSystem.closeAllForUGI(ugi);*{color}
} catch (IOException exception) {
LOG.error("Could not clean up file-system handles for UGI: " + ugi, exception);
}
}
}
Caused by: java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:483)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at org.apache.hadoop.hdfs.DFSClient.getEZForPath(DFSClient.java:2771)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem$54.doCall(DistributedFileSystem.java:2796)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem$54.doCall(DistributedFileSystem.java:2793)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
~[hadoop-common-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.DistributedFileSystem.getEZForPath(DistributedFileSystem.java:2812)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hdfs.client.HdfsAdmin.getEncryptionZoneForPath(HdfsAdmin.java:374)
~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
at
org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.getEncryptionZoneForPath(Hadoop23Shims.java:1384)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
at
org.apache.hadoop.hive.shims.Hadoop23Shims$HdfsEncryptionShim.isPathEncrypted(Hadoop23Shims.java:1379)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.isPathEncrypted(SemanticAnalyzer.java:2484)
~[hive-exec-3.1.3000.7.1.8.11-3.jar:3.1.3000.7.1.8.11-3]
> hive query fails with Filesystem closed error
> -
>
> Key: HIVE-27029
> URL: https://issues.apache.org/jira/browse/HIVE-27029
> Project: Hive
> Issue Type: Bug
>Reporter: Mahesh Raju Somalaraju
>Assignee: Mahesh Raju Somalaraju
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
>
[jira] [Assigned] (HIVE-27055) hive-exec typos part 3
[ https://issues.apache.org/jira/browse/HIVE-27055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michal Lorek reassigned HIVE-27055: --- Assignee: Michal Lorek > hive-exec typos part 3 > -- > > Key: HIVE-27055 > URL: https://issues.apache.org/jira/browse/HIVE-27055 > Project: Hive > Issue Type: Improvement > Components: Query Planning, Query Processor >Affects Versions: 4.0.0 >Reporter: Michal Lorek >Assignee: Michal Lorek >Priority: Trivial > > multiple typos and grammar errors in hive-exec module code and comments -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27054) Backport HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete events
[ https://issues.apache.org/jira/browse/HIVE-27054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh reassigned HIVE-27054: --- Assignee: Indhumathi Muthumurugesh > Backport HIVE-16812 VectorizedOrcAcidRowBatchReader doesn't filter delete > events > > > Key: HIVE-27054 > URL: https://issues.apache.org/jira/browse/HIVE-27054 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Fix For: 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27053) Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in COALESCE / ELT
[ https://issues.apache.org/jira/browse/HIVE-27053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Indhumathi Muthumurugesh reassigned HIVE-27053: --- Assignee: Indhumathi Muthumurugesh > Backport HIVE-20294 Vectorization: Fix NULL / Wrong Results issues in > COALESCE / ELT > > > Key: HIVE-27053 > URL: https://issues.apache.org/jira/browse/HIVE-27053 > Project: Hive > Issue Type: Bug >Reporter: Indhumathi Muthumurugesh >Assignee: Indhumathi Muthumurugesh >Priority: Major > Fix For: 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27052) Backport HIVE-26080: Upgrade accumulo-core to 1.10.1 in branch-3
[ https://issues.apache.org/jira/browse/HIVE-27052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27052: -- Labels: pull-request-available (was: ) > Backport HIVE-26080: Upgrade accumulo-core to 1.10.1 in branch-3 > > > Key: HIVE-27052 > URL: https://issues.apache.org/jira/browse/HIVE-27052 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27052) Backport HIVE-26080: Upgrade accumulo-core to 1.10.1 in branch-3
[ https://issues.apache.org/jira/browse/HIVE-27052?focusedWorklogId=844032=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844032 ] ASF GitHub Bot logged work on HIVE-27052: - Author: ASF GitHub Bot Created on: 07/Feb/23 08:46 Start Date: 07/Feb/23 08:46 Worklog Time Spent: 10m Work Description: Aggarwal-Raghav opened a new pull request, #4034: URL: https://github.com/apache/hive/pull/4034 ### What changes were proposed in this pull request? Upgrade accumulo version to 1.10.1 ### Why are the changes needed? To fix the CVE and be in-sync with master branch ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing Tests Issue Time Tracking --- Worklog Id: (was: 844032) Remaining Estimate: 0h Time Spent: 10m > Backport HIVE-26080: Upgrade accumulo-core to 1.10.1 in branch-3 > > > Key: HIVE-27052 > URL: https://issues.apache.org/jira/browse/HIVE-27052 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27052) Backport HIVE-26080: Upgrade accumulo-core to 1.10.1 in branch-3
[ https://issues.apache.org/jira/browse/HIVE-27052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghav Aggarwal reassigned HIVE-27052: -- > Backport HIVE-26080: Upgrade accumulo-core to 1.10.1 in branch-3 > > > Key: HIVE-27052 > URL: https://issues.apache.org/jira/browse/HIVE-27052 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27051) Backport HIVE-22132: Upgrade commons-lang3 version to 3.9 in branch-3
[ https://issues.apache.org/jira/browse/HIVE-27051?focusedWorklogId=844030=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844030 ] ASF GitHub Bot logged work on HIVE-27051: - Author: ASF GitHub Bot Created on: 07/Feb/23 08:34 Start Date: 07/Feb/23 08:34 Worklog Time Spent: 10m Work Description: Aggarwal-Raghav opened a new pull request, #4033: URL: https://github.com/apache/hive/pull/4033 ### What changes were proposed in this pull request? Upgrade commons-lang3 version from 3.2 -> 3.9 ### Why are the changes needed? To fix the CVE's and be in-sync with master branch ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing tests Issue Time Tracking --- Worklog Id: (was: 844030) Remaining Estimate: 0h Time Spent: 10m > Backport HIVE-22132: Upgrade commons-lang3 version to 3.9 in branch-3 > - > > Key: HIVE-27051 > URL: https://issues.apache.org/jira/browse/HIVE-27051 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27051) Backport HIVE-22132: Upgrade commons-lang3 version to 3.9 in branch-3
[ https://issues.apache.org/jira/browse/HIVE-27051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27051: -- Labels: pull-request-available (was: ) > Backport HIVE-22132: Upgrade commons-lang3 version to 3.9 in branch-3 > - > > Key: HIVE-27051 > URL: https://issues.apache.org/jira/browse/HIVE-27051 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27051) Backport HIVE-22132: Upgrade commons-lang3 version to 3.9 in branch-3
[ https://issues.apache.org/jira/browse/HIVE-27051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghav Aggarwal updated HIVE-27051: --- Summary: Backport HIVE-22132: Upgrade commons-lang3 version to 3.9 in branch-3 (was: Backport HIVE-22132: Upgrade commons-lang3 version to 3.9) > Backport HIVE-22132: Upgrade commons-lang3 version to 3.9 in branch-3 > - > > Key: HIVE-27051 > URL: https://issues.apache.org/jira/browse/HIVE-27051 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27051) Backport HIVE-22132: Upgrade commons-lang3 version to 3.9
[ https://issues.apache.org/jira/browse/HIVE-27051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghav Aggarwal reassigned HIVE-27051: -- > Backport HIVE-22132: Upgrade commons-lang3 version to 3.9 > - > > Key: HIVE-27051 > URL: https://issues.apache.org/jira/browse/HIVE-27051 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-27029) hive query fails with Filesystem closed error
[
https://issues.apache.org/jira/browse/HIVE-27029?focusedWorklogId=844024=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-844024
]
ASF GitHub Bot logged work on HIVE-27029:
-
Author: ASF GitHub Bot
Created on: 07/Feb/23 08:25
Start Date: 07/Feb/23 08:25
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #4024:
URL: https://github.com/apache/hive/pull/4024#issuecomment-1420377331
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=4024)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=BUG)
[1
Bug](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4024=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4024=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=4024=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=CODE_SMELL)
[3 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=4024=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=4024=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=4024=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 844024)
Time Spent: 0.5h (was: 20m)
> hive query fails with Filesystem closed error
> -
>
> Key: HIVE-27029
> URL: https://issues.apache.org/jira/browse/HIVE-27029
> Project: Hive
> Issue Type: Bug
>Reporter: Mahesh Raju Somalaraju
>Assignee: Mahesh Raju Somalaraju
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This Jira is raised to modify/fix the code which is done in part of
> *HIVE-26352.*
>
> we should remove the finally block as this is causing the filesystem close
> errors.
> String queueName, String userName) throws IOException, InterruptedException {
> UserGroupInformation ugi = UserGroupInformation.getCurrentUser();
> try {
> ugi.doAs((PrivilegedExceptionAction) () -> {
> checkQueueAccessInternal(queueName, userName);
> return null;
> });
> } {color:#0747a6}*finally {*{color}
> {color:#0747a6} *try {*{color}
> {color:#0747a6} *FileSystem.closeAllForUGI(ugi);*{color}
> } catch (IOException exception) {
> LOG.error("Could not clean up file-system handles for UGI: " + ugi,
> exception);
> }
> }
> }
>
> Caused by: java.io.IOException: Filesystem closed
> at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:483)
> ~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
> at org.apache.hadoop.hdfs.DFSClient.getEZForPath(DFSClient.java:2771)
> ~[hadoop-hdfs-client-3.1.1.7.1.8.11-3.jar:?]
>
[jira] [Assigned] (HIVE-27049) Iceberg: Provide current snapshot version in show-create-table
[ https://issues.apache.org/jira/browse/HIVE-27049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned HIVE-27049: --- Assignee: Ayush Saxena > Iceberg: Provide current snapshot version in show-create-table > -- > > Key: HIVE-27049 > URL: https://issues.apache.org/jira/browse/HIVE-27049 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: Rajesh Balamohan >Assignee: Ayush Saxena >Priority: Major > > It will be helpful to show "current snapshot" id in "show create table" > statement. This will help in easier debugging. Otherwise, user has to > explicitly query the metadata or read the JSON file to get this info. -- This message was sent by Atlassian Jira (v8.20.10#820010)
