[
https://issues.apache.org/jira/browse/GOBBLIN-2096?focusedWorklogId=923971&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-923971
]
ASF GitHub Bot logged work on GOBBLIN-2096:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 21/Jun/24 14:22
Start Date: 21/Jun/24 14:22
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3982:
URL: https://github.com/apache/gobblin/pull/3982#discussion_r1645195508
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/MysqlMultiActiveLeaseArbiter.java:
##########
@@ -254,7 +256,8 @@ public LeaseAttemptStatus
tryAcquireLease(DagActionStore.DagActionLeaseObject da
log.debug("tryAcquireLease for [{}, is; {}, eventTimestamp: {}] - CASE
1: no existing row for this dag action,"
+ " then go ahead and insert", dagActionLeaseObject.getDagAction(),
dagActionLeaseObject.isReminder() ? "reminder" : "original",
dagActionLeaseObject.getEventTimeMillis());
- int numRowsUpdated =
attemptLeaseIfNewRow(dagActionLeaseObject.getDagAction());
+ int numRowsUpdated =
attemptLeaseIfNewRow(dagActionLeaseObject.getDagAction(),
+
ExponentialBackoff.builder().maxRetries(3).initialDelay(10L).build());
Review Comment:
Can we set the defaults as static variables and include them with units? If
it's just 10ms delay it wouldn't be much more than a jitter
Issue Time Tracking
-------------------
Worklog Id: (was: 923971)
Remaining Estimate: 0h
Time Spent: 10m
> retry Transient SQL Exception for MALA
> ---------------------------------------
>
> Key: GOBBLIN-2096
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2096
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We're seeing a MySQLTransactionRollbackException: Deadlock found when trying
> to get lock when calling MysqlMultiActiveLeaseArbiter. attemptLeaseIfNewRow()
> in the rare case when two hosts check the lease arbiter table at the same
> time, see there's no entry matching the current primary key and try to insert
> it at the exact same time (both wait for lock on the same row). Normally,
> there's slight discrepancy that results in an
> IntegrityConstraintViolationException if one insert succeeds before the other
> OR one finishes the write early and the other sees the entry when reading the
> getResult of the arbiter table. This PR adds retries with exponential backoff
> for transient SQL exceptions to the insert statement. Note, it's not needed
> for other cases when the INSERT/UPDATE statements are conditional on a read
> value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
