[
https://issues.apache.org/jira/browse/HUDI-6186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-6186:
----------------------------
Description:
#6847 extends the `InProcessLockProvider` to support multiple tables in the
same process, by having an in-memory static final map storing the mapping of
the table base path to the read-write reentrant lock, so that the writer uses
the corresponding lock based on the base path. When closing the lock provider,
`close()` removes the lock entry. Since `close()` is called when closing the
write client, the lock is removed and subsequent concurrent writers will get a
different lock instance on the same table, causing the locking mechanism on the
same table to be useless. Take the following example where three writers write
to the same table concurrently and need to acquire the in-process lock:
```
Writer 1: lock |----------------| unlock and close
Writer 2: try lock | ... lock |------| unlock and close
Writer 3: try lock | ... lock |------|
unlock and close
```
after Writer 1 releases the lock and closes the lock provider, the lock
instance is removed from the map, and Writer 3 will get a different lock
instance compared to Writer 2.
> Fix lock identity in InProcessLockProvider
> ------------------------------------------
>
> Key: HUDI-6186
> URL: https://issues.apache.org/jira/browse/HUDI-6186
> Project: Apache Hudi
> Issue Type: Bug
> Components: multi-writer
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Fix For: 0.13.1
>
>
> #6847 extends the `InProcessLockProvider` to support multiple tables in the
> same process, by having an in-memory static final map storing the mapping of
> the table base path to the read-write reentrant lock, so that the writer uses
> the corresponding lock based on the base path. When closing the lock
> provider, `close()` removes the lock entry. Since `close()` is called when
> closing the write client, the lock is removed and subsequent concurrent
> writers will get a different lock instance on the same table, causing the
> locking mechanism on the same table to be useless. Take the following
> example where three writers write to the same table concurrently and need to
> acquire the in-process lock:
> ```
> Writer 1: lock |----------------| unlock and close
> Writer 2: try lock | ... lock |------| unlock and close
> Writer 3: try lock | ... lock |------|
> unlock and close
> ```
> after Writer 1 releases the lock and closes the lock provider, the lock
> instance is removed from the map, and Writer 3 will get a different lock
> instance compared to Writer 2.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
