sivabalan narayanan created HUDI-9247:
-----------------------------------------
Summary: Re-evaluate reuse of TimeGenerator instance
Key: HUDI-9247
URL: https://issues.apache.org/jira/browse/HUDI-9247
Project: Apache Hudi
Issue Type: Improvement
Components: writer-core
Reporter: sivabalan narayanan
To acquire lock to generate new instant time, we use TimeGenerators apis.
[https://github.com/apache/hudi/blob/93b4da301d12b4a2d66787d80ce3071876aaad22/hudi-common/src/main/java/org/apache/hudi/common/table/timeline/TimeGenerators.java#L45]
We have a `reuse` config which on enabled, multiple invocations will end up
re-using the same instance. For every other lock provider, this might work out
since it relies on remote calls (zookeeper, hms, dynamo db etc). But for
InProcessLock provider, when multiple threads end up using same instance of
`InProcessLockProvider`, unlock by thread1 might actually impact the tryLock
call by a different thread.
So, lets take a look at diff lock providers available and only allow re-use (
_hoodie.time.generator.reuse.enable) only for those that makes sense.
For eg, InProcessLockProvider, ConditionalWritesLockProvider, we should not
allow re-use. And make the default value false as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
