danny0405 commented on code in PR #11347:
URL: https://github.com/apache/hudi/pull/11347#discussion_r1618425792
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/transaction/lock/LockManager.java:
##########
@@ -81,12 +85,16 @@ public void lock() {
metrics.startLockApiTimerContext();
if
(!getLockProvider().tryLock(writeConfig.getLockAcquireWaitTimeoutInMs(),
TimeUnit.MILLISECONDS)) {
metrics.updateLockNotAcquiredMetric();
+ if (getLockProvider() instanceof FileSystemBasedLockProvider) {
+ fs.delete(new Path(((FileSystemBasedLockProvider)
getLockProvider()).getLock()), true);
Review Comment:
Like @KnightChess said, could we add some timeout check when fetching the
filesystem lock, let's say within the fs lock provider:
1. try to fetch the lock;
1.1 check if there is already a lock file and check the modification time
for it;
1.1.1 if the time expires, just fetch the lock;
1.1.2 if not, remove the legacy lock file and create a new one (still,
here the lock may sneak out by other fetchers, but the file creation would
throw exception, so we are okay)
1.2 if there is no expiry lock file, just creates a new one
Another critical thing we need to care is that only the object storage
provides atomicy for file creation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
