[
https://issues.apache.org/jira/browse/GOBBLIN-697?focusedWorklogId=212192&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-212192
]
ASF GitHub Bot logged work on GOBBLIN-697:
------------------------------------------
Author: ASF GitHub Bot
Created on: 13/Mar/19 02:40
Start Date: 13/Mar/19 02:40
Worklog Time Spent: 10m
Work Description: yukuai518 commented on pull request #2568:
[GOBBLIN-697] Implementation of data file versioning and preservation in distcp.
URL: https://github.com/apache/incubator-gobblin/pull/2568#discussion_r264956432
##########
File path:
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/publisher/CopyDataPublisher.java
##########
@@ -119,6 +125,12 @@ public CopyDataPublisher(State state) throws IOException {
this.recoveryHelper = new RecoveryHelper(this.fs, state);
this.recoveryHelper.purgeOldPersistedFile();
+
+ Config config = ConfigUtils.propertiesToConfig(state.getProperties());
+
+ this.srcDataFileVersion = DataFileVersionStrategy
Review comment:
this should be this.srcDataFileVersionStrategy?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 212192)
Time Spent: 1h 40m (was: 1.5h)
> Allow distcp to carry over file version independently of modtime
> ----------------------------------------------------------------
>
> Key: GOBBLIN-697
> URL: https://issues.apache.org/jira/browse/GOBBLIN-697
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Issac Buenrostro
> Assignee: Issac Buenrostro
> Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Examples where this might be useful is data syncing between two locations.
> Relying on modification times to detect data changes may lead to a feedback
> loop of copying: data gets created at location A at time 0, at time 1 data is
> copied to location B, sync mechanism might incorrectly believe that since mod
> time of location B is higher, it should be synced back to location A, etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
