Zihan Li created GOBBLIN-1778:
---------------------------------
Summary: Add house keeping thread in DagManager to periodically
sync in memory state with mysql table
Key: GOBBLIN-1778
URL: https://issues.apache.org/jira/browse/GOBBLIN-1778
Project: Apache Gobblin
Issue Type: Improvement
Reporter: Zihan Li
Now dag managers have the assumption that it is the only process that can
update mysql table and the in-memory state is always in sync with mysql. But we
do notice that during the leader transforms period, it's possible that two dag
manager can run concurrently and update the mysql db at the same time.
To address that, we need either add a lock to make sure only one dag manager is
working at one time, or we need to have a housekeeping thread to periodically
sync the in-memory state with the mysql table. After discussion, we choose to
go with later approach
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
