hudi-bot opened a new issue, #16943:
URL: https://github.com/apache/hudi/issues/16943
Only InProcessLockProvider is thread safe as of now. The other lock
providers are not thread safe. The Jira aims to ensure all the lock providers
which support thread safety are made thread safe.
- InProcessLockProvider uses ReentrantReadWriteLock which inherently is
thread safe.
- DynamoDBBasedLockProvider uses AmazonDynamoDBLockClient which is thread
safe but it stores the lock item as an instance variable which can cause issues
with multiple threads.
- Similarly ZookeeperBasedLockProvider uses CuratorFramework which is
thread safe but it stores InterProcessMutex as instance variable. The
InterProcessMutex is then released during unlock.
- HiveMetastoreBasedLockProvider uses IMetaStoreClient which inherently is
not thread safe.
- FileSystemBasedLockProvider is also not thread safe since it stores
LockInfo and currentOwnerLockInfo as instance variables.
Currently lock provider is closed during unlock and if we are sharing the
lock provider amongst multiple threads, we would need to fix the close
semantics for lock provider as well.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-9254
- Type: Bug
- Fix version(s):
- 1.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
