Ma Jian created HUDI-8109:
-----------------------------
Summary: Fix the error related to locking during
clustering/compaction when calling the file system in OSS.
Key: HUDI-8109
URL: https://issues.apache.org/jira/browse/HUDI-8109
Project: Apache Hudi
Issue Type: Bug
Reporter: Ma Jian
When a table is on OSS and MDT is enabled, the current logic adds some lock
parameters. It sets the {{FileSystemBasedLockProvider}} as the default lock.
However, in its constructor, it checks if the current file system supports
atomicity. If it does not, an error is thrown. Currently, it seems there is no
configuration to bypass this error, which prevents tables on OSS from using
capabilities like calling clustering procedures.
The fix I implemented is to skip adding this configuration when the file system
does not support atomicity. This way, at least the basic functionality won't be
blocked by this lock and can operate normally.
The error message is as follows:
Exception in thread "main" org.apache.hudi.exception.HoodieException: Unable to
instantiate class
org.apache.hudi.client.transaction.lock.FileSystemBasedLockProvider
at
org.apache.hudi.common.util.ReflectionUtils.loadClass(ReflectionUtils.java:75)
at
org.apache.hudi.client.transaction.lock.LockManager.getLockProvider(LockManager.java:125)
at
org.apache.hudi.client.transaction.lock.LockManager.unlock(LockManager.java:112)
at
org.apache.hudi.client.transaction.TransactionManager.endTransaction(TransactionManager.java:70)
at
org.apache.hudi.client.BaseHoodieTableServiceClient.scheduleTableService(BaseHoodieTableServiceClient.java:609)
at
org.apache.hudi.client.BaseHoodieWriteClient.scheduleTableService(BaseHoodieWriteClient.java:1216)
at
org.apache.hudi.client.BaseHoodieWriteClient.scheduleClusteringAtInstant(BaseHoodieWriteClient.java:1165)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
