[
https://issues.apache.org/jira/browse/HIVE-26012?focusedWorklogId=799664&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-799664
]
ASF GitHub Bot logged work on HIVE-26012:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Aug/22 09:16
Start Date: 10/Aug/22 09:16
Worklog Time Spent: 10m
Work Description: DanielZhu58 commented on code in PR #3477:
URL: https://github.com/apache/hive/pull/3477#discussion_r942219606
##########
standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift:
##########
@@ -901,7 +901,9 @@ struct AddPartitionsRequest {
4: required bool ifNotExists,
5: optional bool needResult=true,
6: optional string catName,
- 7: optional string validWriteIdList
+ 7: optional string validWriteIdList,
+ // use boolean skipFSWrites to decide whether create directory in file
system or not
+ 8: optional bool skipFSWrites=false
Review Comment:
Yes, we do. Because that's one of the requirement of this patch.
##########
standalone-metastore/metastore-common/src/main/thrift/hive_metastore.thrift:
##########
@@ -2040,7 +2044,10 @@ struct CreateDatabaseRequest {
9: optional i32 createTime,
10: optional string managedLocationUri,
11: optional string type,
- 12: optional string dataConnectorName
+ 12: optional string dataConnectorName,
+ // use boolean skipFSWrites to decide whether create directory in file
system or not
+ 13: optional bool skipFSWrites=false,
+ 14: optional Database database
Review Comment:
Yes, we do. Because in the create database function with signature
CreateDatabaseRequest, we will use getDatabase() function. This will need the
`optional Database database`.
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java:
##########
@@ -1229,74 +1229,78 @@ private void create_database_core(RawStore ms, final
Database db)
Map<String, String> transactionalListenersResponses =
Collections.emptyMap();
try {
firePreEvent(new PreCreateDatabaseEvent(db, this));
- //reinstate location uri for metastore db.
- if (skipAuthorization == true){
Review Comment:
We can leave it the same way this time. Because this change is not highly
related with the HMS directory skipping.
Issue Time Tracking
-------------------
Worklog Id: (was: 799664)
Time Spent: 3h 10m (was: 3h)
> HMS APIs to be enhanced for metadata replication
> ------------------------------------------------
>
> Key: HIVE-26012
> URL: https://issues.apache.org/jira/browse/HIVE-26012
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 3.1.0
> Reporter: Naveen Gangam
> Assignee: Hongdan Zhu
> Priority: Major
> Labels: pull-request-available
> Attachments: HMS APIs to be enhanced for metadata replication.docx
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> HMS currently has APIs like these that automatically create/delete the
> directories on the associated DFS.
> [create/drop]_database
> [create/drop]_table*
> [add/append/drop]_partition*
> This is expected and should be this way when query processors use this APIs.
> However, when tools that replicate hive metadata use this APIs on the target
> cluster, creating these dirs on target side which cause the replication of
> DFS-snapshots to fail.
> So we if provide an option to bypass this creation of dirs, dfs replications
> will be smoother. In the future we will need to restrict users that can use
> these APIs. So we will have some sort of an authorization policy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
