wuchong commented on PR #1477:
URL: https://github.com/apache/fluss/pull/1477#issuecomment-3269056684
Thank you for the response, @LiebingYu. I understand the context of this
issue and would like to suggest the following improvements:
- It seems the default executor service of curator `inBackground()` uses a
single-threaded executor, which could become a performance bottleneck under a
high volume of partition deletion requests. To address this, we can leverage
the coordinator’s `ioExecutor` to handle these operations asynchronously, which
is better suited for I/O-bound tasks.
- We already perform the deletion of remote data directories asynchronously
in `TableManager#completeDeletePartition`. We should extend this pattern by
also making the deletion of ZK assignment asynchronous, either in
`TableManager#completeDeletePartition` or
`MetadataManager#completeDeletePartition`.
- Currently, even if the ZK assignment deletion fails, we proceed to remove
the partition from `CoordinatorContext`. Given this behavior, making the ZK
deletion asynchronous should not introduce additional risk, as the system
already assumes best-effort cleanup.
- For consistency, we should apply the same async pattern to
`MetadataManager#completeDeleteTable`.
- To implement this, we can pass the `ioExecutor` into `MetadataManager` for
executing the async operations, without requiring any changes to the
`ZookeeperClient` itself.
Let me know what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
