xiaochen-zhou commented on PR #1507: URL: https://github.com/apache/fluss/pull/1507#issuecomment-3173045426
> Thanks @xiaochen-zhou for the contribution. I have some comments for the PR. > > Please create an issue and reach consensus on the approach before submitting a pull request. > > While this PR does enhance the code, I'm not sure it will significantly reduce network load in practice. The listPartitions call is made every 10 seconds, and typically there are fewer than 100 partitions in total — so the payload size is relatively small. > > More importantly, the main bottleneck for this RPC lies in ZooKeeper performance, and since the partition filtering isn't pushed down to the ZooKeeper layer, the server still retrieves all partition data before applying the filter. As a result, we don’t get the expected reduction in backend load. I think we can optimize on the ZK side: when the ResolvedPartitionSpec is a complete partition spec, we can directly retrieve the partition instead of scanning all partitions. This can satisfy some scenarios. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
