nsivabalan commented on PR #13077: URL: https://github.com/apache/hudi/pull/13077#issuecomment-2776502799
but looking at your PR desc, if only InProcessLockProvider is eligible for re-use, I am wondering, why can't we disallow re-use only for any lock provider. anyways incase of other lock providers, looks like re-use is not possible. Incase of InProcess lock provider, for a given table, we might have max of 4 to 5 concurrent threads (ingestion writer , and 3 table services) which might be looking to generate instant time. And InProcessLockProvider internally ensures all of them will end up using the same instance of ReentrantReadWriteLock for a given table. So, we are not gaining much by allowing re-use for InProcessLockProvider and there is not a lot of heavy weight resources we are starting up. Its going to be an issue w/ other lock provider where we might have client objects establishing connection to external endpoints, but looks like all the lock providers need to be worked upon to make it threadsafe. Based on this, I feel we can disallow re-use only for now and think through methodically how to go about this and rush through 1.0.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
