[ 
https://issues.apache.org/jira/browse/GOBBLIN-1984?focusedWorklogId=899525&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-899525
 ]

ASF GitHub Bot logged work on GOBBLIN-1984:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 12/Jan/24 22:45
            Start Date: 12/Jan/24 22:45
    Worklog Time Spent: 10m 
      Work Description: phet commented on code in PR #3857:
URL: https://github.com/apache/gobblin/pull/3857#discussion_r1450988687


##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/api/FlowSpec.java:
##########
@@ -75,13 +78,22 @@ public class FlowSpec implements Configurable, Spec {
   /** Human-readable description of the flow spec */
   final String description;
 
-  /** Flow config as a typesafe config object */
-  final Config config;
+  /* Note that since getConfig() and getConfigAsProperties() are independent 
accessors, `volatile` doesn't ensure a
+  * consistent view between them. In a multi-threaded scenario one should use 
the following access mechanism:
+  *    FlowSpec fs = ...
+  *    synchronized (fs) {

Review Comment:
   I do agree it would be overkill in the general case, wherein the user is 
only getting one or the other of the two.  there's no need for a single 
accessor to synchronize on the instance, given `volatile`.
   
   what I believe urmi's suggesting is to lock the object if one wants to 
access both.  this ensures both sides agree.  perhaps such clarification could 
be added to the comment:
   ```
   In a multi-threaded scenario one should use the following access mechanism:
   ```
   ?
   
   it sounds overly broad as phrased above, given "a MT scenario" applies to 
all our code everywhere.
   
   the other important thing to mention is that the `synchronized` block should 
only be small enough to obtain the two, so it doesn't keep the object locked.
   
   the other alternative would be to add a combo method to this `FlowSpec` 
derived class:
   ```
   /** use for consistency between the two... */
   public synchronized Pair<Config, Properties> getConfigAndProperties() {
     // ultimate correctness would be to return a defensive copy of `Properties`
   }
   ```
   would this perhaps be clearest advice to give?





Issue Time Tracking
-------------------

    Worklog Id:     (was: 899525)
    Time Spent: 5.5h  (was: 5h 20m)

> Add Consensus FlowExecutionId to FlowSpec used for Compilation and Tracking 
> Dag
> -------------------------------------------------------------------------------
>
>                 Key: GOBBLIN-1984
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1984
>             Project: Apache Gobblin
>          Issue Type: Bug
>          Components: gobblin-service
>            Reporter: Urmi Mustafi
>            Assignee: Abhishek Tiwari
>            Priority: Major
>          Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> In a previous change, we noticed the compiled flowExecutionId was different 
> from the one during execution and tried to fix it by updating the 
> flowMetadata used to send GTE relating to compilation. We succeeded in 
> updating the compilation event to use the flowExecutionId but did not update 
> the FlowSpec used by Compiler to create the Dag which is used during the 
> execution phase of flow management. The compiler retrieves a flowExecutionId 
> if one exists in the flowSpec (which it does for adhoc flows), otherwise 
> generates a new one 
> ([code|[https://jarvis.corp.linkedin.com/codesearch/result/?name=MultiHopFlowCompiler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Fflow&reponame=linkedin%2Fgobblin-elr#301]).]
>  We do need to modify the FlowSpec then with the consensus flowExecutionId 
> before forwarding it to the DagManager.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to