[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=608538=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-608538
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 08/Jun/21 15:41
Start Date: 08/Jun/21 15:41
Worklog Time Spent: 10m
Work Description: nrg4878 closed pull request #2354:
URL: https://github.com/apache/hive/pull/2354
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 608538)
Time Spent: 1.5h (was: 1h 20m)
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=608537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-608537
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 08/Jun/21 15:41
Start Date: 08/Jun/21 15:41
Worklog Time Spent: 10m
Work Description: nrg4878 commented on pull request #2354:
URL: https://github.com/apache/hive/pull/2354#issuecomment-856881562
Thanks for the review. Fix has been committed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 608537)
Time Spent: 1h 20m (was: 1h 10m)
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=607501=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-607501
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 06/Jun/21 15:49
Start Date: 06/Jun/21 15:49
Worklog Time Spent: 10m
Work Description: nrg4878 opened a new pull request #2354:
URL: https://github.com/apache/hive/pull/2354
### What changes were proposed in this pull request?
The database's managed location is not deleted when the database is dropped.
This fix addresses the issue.
### Why are the changes needed?
Even though db.getManagedLocation() might be null, there is always a default
managed location for the database. On a drop database, we check to see if this
location is null and only drop it if not null. But regardless of whether or not
it is null, there is always an assigned location that needs to be dropped.
### Does this PR introduce _any_ user-facing change?
NO
### How was this patch tested?
Unit test + manually
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 607501)
Time Spent: 1h 10m (was: 1h)
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=606444=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-606444
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 04/Jun/21 07:44
Start Date: 04/Jun/21 07:44
Worklog Time Spent: 10m
Work Description: nrg4878 commented on a change in pull request #1506:
URL: https://github.com/apache/hive/pull/1506#discussion_r644869934
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##
@@ -1917,6 +1923,12 @@ private void drop_database_core(RawStore ms, String
catName,
LOG.error("Failed to delete database directory: " +
db.getLocationUri() +
" " + e.getMessage());
}
+ try {
+wh.deleteDir(wh.getDatabaseManagedPath(db), true, db);
+ } catch (Exception e) {
+LOG.error("Failed to delete database directory: " +
db.getLocationUri() +
Review comment:
I will rebase this patch. All this code has been refactored into
HMSHandler.java. I will have to redo this fix. I will address it then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 606444)
Time Spent: 1h (was: 50m)
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=605976=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-605976
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 03/Jun/21 15:00
Start Date: 03/Jun/21 15:00
Worklog Time Spent: 10m
Work Description: nrg4878 commented on a change in pull request #1506:
URL: https://github.com/apache/hive/pull/1506#discussion_r644869934
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##
@@ -1917,6 +1923,12 @@ private void drop_database_core(RawStore ms, String
catName,
LOG.error("Failed to delete database directory: " +
db.getLocationUri() +
" " + e.getMessage());
}
+ try {
+wh.deleteDir(wh.getDatabaseManagedPath(db), true, db);
+ } catch (Exception e) {
+LOG.error("Failed to delete database directory: " +
db.getLocationUri() +
Review comment:
I will rebase this patch. All this code has been refactored into
HMSHandler.java. I will have to redo this fix. I will address it then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 605976)
Time Spent: 50m (was: 40m)
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=603227=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-603227
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 27/May/21 19:30
Start Date: 27/May/21 19:30
Worklog Time Spent: 10m
Work Description: yongzhi commented on a change in pull request #1506:
URL: https://github.com/apache/hive/pull/1506#discussion_r640908620
##
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
##
@@ -1917,6 +1923,12 @@ private void drop_database_core(RawStore ms, String
catName,
LOG.error("Failed to delete database directory: " +
db.getLocationUri() +
" " + e.getMessage());
}
+ try {
+wh.deleteDir(wh.getDatabaseManagedPath(db), true, db);
+ } catch (Exception e) {
+LOG.error("Failed to delete database directory: " +
db.getLocationUri() +
Review comment:
should we are more consistent with the delete behavior, throw exception
here too (the same as line 1789), the same is for 1923
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 603227)
Time Spent: 40m (was: 0.5h)
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=516412=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-516412
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 25/Nov/20 00:43
Start Date: 25/Nov/20 00:43
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #1506:
URL: https://github.com/apache/hive/pull/1506
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 516412)
Time Spent: 0.5h (was: 20m)
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=513251=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-513251
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 18/Nov/20 00:40
Start Date: 18/Nov/20 00:40
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #1506:
URL: https://github.com/apache/hive/pull/1506#issuecomment-729301314
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 513251)
Time Spent: 20m (was: 10m)
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24135) Drop database doesn't delete directory in managed location
[
https://issues.apache.org/jira/browse/HIVE-24135?focusedWorklogId=486057=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-486057
]
ASF GitHub Bot logged work on HIVE-24135:
-
Author: ASF GitHub Bot
Created on: 18/Sep/20 04:14
Start Date: 18/Sep/20 04:14
Worklog Time Spent: 10m
Work Description: nrg4878 opened a new pull request #1506:
URL: https://github.com/apache/hive/pull/1506
…n (Naveen Gangam)
### What changes were proposed in this pull request?
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 486057)
Remaining Estimate: 0h
Time Spent: 10m
> Drop database doesn't delete directory in managed location
> --
>
> Key: HIVE-24135
> URL: https://issues.apache.org/jira/browse/HIVE-24135
> Project: Hive
> Issue Type: Sub-task
>Reporter: Karen Coppage
>Assignee: Naveen Gangam
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Repro:
> say the default managed location is managed/hive and the default external
> location is external/hive.
> {code:java}
> create database db1; -- creates: external/hive/db1.db
> create table db1.table1 (i int); -- creates: managed/hive/db1.db and
> managed/hive/db1.db/table1
> drop database db1 cascade; -- removes : external/hive/db1.db and
> managed/hive/db1.db/table1
> {code}
> Problem: Directory managed/hive/db1.db remains.
> Since HIVE-22995, dbs have a managed (managedLocationUri) and an external
> location (locationUri). I think the issue is that
> HiveMetaStore.HMSHandler#drop_database_core deletes only the db directory in
> the external location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
