[
https://issues.apache.org/jira/browse/GOBBLIN-1778?focusedWorklogId=843332&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843332
]
ASF GitHub Bot logged work on GOBBLIN-1778:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 02/Feb/23 23:18
Start Date: 02/Feb/23 23:18
Worklog Time Spent: 10m
Work Description: codecov-commenter commented on PR #3635:
URL: https://github.com/apache/gobblin/pull/3635#issuecomment-1414501865
#
[Codecov](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
Report
> Merging
[#3635](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
(9e80276) into
[master](https://codecov.io/gh/apache/gobblin/commit/9f8ab24c8cf4bc195d23ebac0448a07ce2f91e2f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
(9f8ab24) will **increase** coverage by `3.63%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #3635 +/- ##
============================================
+ Coverage 40.16% 43.79% +3.63%
+ Complexity 3544 2063 -1481
============================================
Files 791 409 -382
Lines 33285 17639 -15646
Branches 3699 2152 -1547
============================================
- Hits 13368 7725 -5643
+ Misses 18601 9056 -9545
+ Partials 1316 858 -458
```
| [Impacted
Files](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
| Coverage Δ | |
|---|---|---|
|
[.../gobblin/cluster/GobblinHelixTaskStateTracker.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkhlbGl4VGFza1N0YXRlVHJhY2tlci5qYXZh)
| `62.50% <0.00%> (-6.25%)` | :arrow_down: |
|
[...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh)
| `44.44% <0.00%> (-1.67%)` | :arrow_down: |
|
[...ion/google/webmaster/UrlTriePostOrderIterator.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvb2dsZS1pbmdlc3Rpb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vaW5nZXN0aW9uL2dvb2dsZS93ZWJtYXN0ZXIvVXJsVHJpZVBvc3RPcmRlckl0ZXJhdG9yLmphdmE=)
| | |
|
[...che/gobblin/metrics/influxdb/InfluxDBReporter.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tbWV0cmljcy1pbmZsdXhkYi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9tZXRyaWNzL2luZmx1eGRiL0luZmx1eERCUmVwb3J0ZXIuamF2YQ==)
| | |
|
[...apache/gobblin/kafka/writer/KafkaWriterHelper.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2thZmthL3dyaXRlci9LYWZrYVdyaXRlckhlbHBlci5qYXZh)
| | |
|
[...org/apache/gobblin/crypto/HexKeyToStringCodec.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY3J5cHRvL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NyeXB0by9IZXhLZXlUb1N0cmluZ0NvZGVjLmphdmE=)
| | |
|
[...apache/gobblin/couchbase/common/TupleDocument.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY291Y2hiYXNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvdWNoYmFzZS9jb21tb24vVHVwbGVEb2N1bWVudC5qYXZh)
| | |
|
[...ava/org/apache/gobblin/http/ApacheHttpRequest.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4taHR0cC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9odHRwL0FwYWNoZUh0dHBSZXF1ZXN0LmphdmE=)
| | |
|
[...ecordToEncryptedSerializedRecordConverterBase.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY3J5cHRvL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9TZXJpYWxpemVkUmVjb3JkVG9FbmNyeXB0ZWRTZXJpYWxpemVkUmVjb3JkQ29udmVydGVyQmFzZS5qYXZh)
| | |
|
[...extract/kafka/PreviousOffsetNotFoundException.java](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9rYWZrYS9QcmV2aW91c09mZnNldE5vdEZvdW5kRXhjZXB0aW9uLmphdmE=)
| | |
| ... and [375
more](https://codecov.io/gh/apache/gobblin/pull/3635?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
| |
:mega: We’re building smart automated test selection to slash your CI/CD
build times. [Learn
more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
Issue Time Tracking
-------------------
Worklog Id: (was: 843332)
Time Spent: 20m (was: 10m)
> Add house keeping thread in DagManager to periodically sync in memory state
> with mysql table
> ---------------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1778
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1778
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Zihan Li
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Now dag managers have the assumption that it is the only process that can
> update mysql table and the in-memory state is always in sync with mysql. But
> we do notice that during the leader transforms period, it's possible that two
> dag manager can run concurrently and update the mysql db at the same time.
> To address that, we need either add a lock to make sure only one dag manager
> is working at one time, or we need to have a housekeeping thread to
> periodically sync the in-memory state with the mysql table. After discussion,
> we choose to go with later approach
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
