[
https://issues.apache.org/jira/browse/GOBBLIN-1696?focusedWorklogId=811512&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-811512
]
ASF GitHub Bot logged work on GOBBLIN-1696:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 23/Sep/22 07:28
Start Date: 23/Sep/22 07:28
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3548:
URL: https://github.com/apache/gobblin/pull/3548#discussion_r978337541
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/flowgraph/GitFlowGraphListener.java:
##########
@@ -53,10 +47,10 @@ public GitFlowGraphListener(Optional<? extends
FSFlowTemplateCatalog> flowTempla
@Override
public void addChange(DiffEntry change) {
Path path = new Path(change.getNewPath());
- if (path.depth() == NODE_FILE_DEPTH) {
- addDataNode(change.getNewPath());
- } else if (path.depth() == EDGE_FILE_DEPTH) {
- addFlowEdge(change.getNewPath());
+ if (path.depth() == BaseFlowGraphHelper.NODE_FILE_DEPTH) {
+ this.baseFlowGraphHelper.addDataNode(this.flowGraph.get(),
change.getNewPath());
Review Comment:
I'm going to lean the latter approach which seems simpler to implement.
Reason being is that I don't think that it is possible for the Git Flowgraph to
run into issues with concurrent updates given the use of it (though it is
theoretical in practice) and also due to that this entire module will be
deprecated soon anyways due to the FSFlowgraph.
Issue Time Tracking
-------------------
Worklog Id: (was: 811512)
Time Spent: 4h 10m (was: 4h)
> Build a file-based flowgraph that watches for changes and updates
> -----------------------------------------------------------------
>
> Key: GOBBLIN-1696
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1696
> Project: Apache Gobblin
> Issue Type: New Feature
> Components: gobblin-service
> Reporter: William Lo
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Gobblin-as-a-Service only has a Git based flowgraph, which is difficult to
> build CI/CD around. We can provide an alternate flowgraph that is just based
> off files. This flowgraph should update atomically.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
