[
https://issues.apache.org/jira/browse/GOBBLIN-1720?focusedWorklogId=814850&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-814850
]
ASF GitHub Bot logged work on GOBBLIN-1720:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 08/Oct/22 00:53
Start Date: 08/Oct/22 00:53
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3577:
URL: https://github.com/apache/gobblin/pull/3577#discussion_r990564886
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/iceberg/IcebergDataset.java:
##########
@@ -137,13 +139,17 @@ Collection<CopyEntity> generateCopyEntities(FileSystem
targetFs, CopyConfigurati
// TODO: determine whether unnecessarily expensive to repeatedly
re-create what should be the same FS: could it
// instead be created once and reused thereafter?
FileSystem actualSourceFs =
getSourceFileSystemFromFileStatus(srcFileStatus, defaultHadoopConfiguration);
+ Path toPath = PathUtils.getRootPathParent(srcPath);
Review Comment:
I'm unsure if we need to recursively ensure that all permissions are the
same until root, I think this can cause issues cross cluster. At least with
hive copy, we only validate the table path and its parent (not saying that this
is the most robust way of doing it though, we've seen issues with that approach
too).
I think the safest and most predictable approach is to ensure that all the
directories that Gobblin will create will have the same permissions as the
source.
Issue Time Tracking
-------------------
Worklog Id: (was: 814850)
Time Spent: 0.5h (was: 20m)
> Preserve Ancestor Owner and Permissions for Fs between Src and Dest for
> Iceberg Distcp
> --------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1720
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1720
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Meeth Gala
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> We want to preserve the Fs ownership and permissions between src and dest
> while performing an Iceberg based distcp. Currently, we are preserving all
> the permissions up to root dir for Iceberg tables.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
