[
https://issues.apache.org/jira/browse/HUDI-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-2573:
--------------------------------------
Description:
With synchronous metadata patch, we added locking for cleaning and rollbacks.
but there are code paths, where we do double locking and hence it hangs or
fails after sometime.
inline cleaning enabled.
C1 acquires lock.
post commit -> triggers cleaning.
cleaning again tries to acquire lock when about to commit
and this is problematic.
Also, when upgrade is needed, we take a lock and rollback failed writes. this
again will run into issues w/ double locking.
was:
With synchronous metadata patch, we added locking for cleaning and rollbacks.
but there are code paths, where we do double locking and hence it hangs or
fails after sometime.
inline cleaning enabled.
```
C1 acquires lock.
post commit -> triggers cleaning.
cleaning again tries to acquire lock when about to commit
and this is problematic.
```
Also, when upgrade is needed, we take a lock and rollback failed writes. this
again will run into issues w/ double locking.
> Deadlock w/ multi writer
> -------------------------
>
> Key: HUDI-2573
> URL: https://issues.apache.org/jira/browse/HUDI-2573
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
> Fix For: 0.10.0
>
>
> With synchronous metadata patch, we added locking for cleaning and rollbacks.
> but there are code paths, where we do double locking and hence it hangs or
> fails after sometime.
>
> inline cleaning enabled.
>
> C1 acquires lock.
> post commit -> triggers cleaning.
> cleaning again tries to acquire lock when about to
> commit and this is problematic.
>
> Also, when upgrade is needed, we take a lock and rollback failed writes. this
> again will run into issues w/ double locking.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
