[
https://issues.apache.org/jira/browse/GOBBLIN-2049?focusedWorklogId=915422&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-915422
]
ASF GitHub Bot logged work on GOBBLIN-2049:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 18/Apr/24 23:44
Start Date: 18/Apr/24 23:44
Worklog Time Spent: 10m
Work Description: umustafi commented on code in PR #3929:
URL: https://github.com/apache/gobblin/pull/3929#discussion_r1571517177
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/writer/FileAwareInputStreamDataWriter.java:
##########
@@ -90,6 +90,8 @@ public class FileAwareInputStreamDataWriter extends
InstrumentedDataWriter<FileA
public static final boolean DEFAULT_GOBBLIN_COPY_CHECK_FILESIZE = false;
public static final String GOBBLIN_COPY_TASK_OVERWRITE_ON_COMMIT =
"gobblin.copy.task.overwrite.on.commit";
public static final boolean DEFAULT_GOBBLIN_COPY_TASK_OVERWRITE_ON_COMMIT =
false;
+ public static final String GOBBLIN_COPY_REQUIRE_PERMISSION_SET_FOR_SUCCESS =
"gobblin.copy.requirePermissionSetForSuccess";
+ public static final boolean DEFAULT_COPY_REQUIRE_PERMISSION_SET_FOR_SUCCESS
= false;
Review Comment:
Sure, I was also debating making the default true instead. I'll change.
Issue Time Tracking
-------------------
Worklog Id: (was: 915422)
Time Spent: 50m (was: 40m)
> Configure Gobblin Distcp Writer to fail if setPermission fails
> --------------------------------------------------------------
>
> Key: GOBBLIN-2049
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2049
> Project: Apache Gobblin
> Issue Type: New Feature
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Gobblin {{safeSetPathPermission}} does not throw an exception when setting
> permissions fail on a path. We want to change this behavior especially for
> use cases like manifest distcp where hundreds of thousands of files are
> involved in a distcp job and permission settings are important to replicate
> correctly as they cannot be updated or verified by hand.
> This PR adds a new configuration to fail writing or publishing tasks when the
> job is configured to report success when permissions are not replicated
> properly. The default behavior remains the same as before to allow the job to
> succeed without this for backwards compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
