Re: [Discuss][PIP-164] Support split bundle by flow or qps

2022-08-23 Thread PengHui Li
Thanks for the explanation LGTM Thanks, Penghui On Tue, Aug 23, 2022 at 11:41 PM Dave Fisher wrote: > While not a comment about this proposal I have a comment another split > bundle concept. > > Manually split out a topic into its own bundle. > > Say a bundle is 0x to 0x0200 with 5

Re: [Discuss][PIP-164] Support split bundle by flow or qps

2022-08-23 Thread Dave Fisher
While not a comment about this proposal I have a comment another split bundle concept. Manually split out a topic into its own bundle. Say a bundle is 0x to 0x0200 with 5 topics. t1 at 0x0010 t2 at 0x0030 t3 at 0x0110 t4 at 0x0123 t5 at 0x01AE Let’s split out

??????[Discuss][PIP-164] Support split bundle by flow or qps

2022-08-23 Thread lordcheng10
"It's a good idea to improve the bundle split for the case that the traffic of the topic doesn't change drastically Otherwise, we should not use this policy. or can we use it for all cases?" 1.It is suitable for scenarios where topic traffic is relatively stable. In addition, we can also adjust

?????? [Discuss][PIP-164] Support split bundle by flow or qps

2022-08-23 Thread lordcheng10
It's a good idea to improve the bundle split for the case that the traffic of the topic doesn't change drastically Otherwise, we should not use this policy. or can we use it for all cases? 1.It is suitable for scenarios where topic traffic is relatively stable. In addition, we can also adjust

Re: [Discuss][PIP-164] Support split bundle by flow or qps

2022-08-23 Thread PengHui Li
It's a good idea to improve the bundle split for the case that the traffic of the topic doesn't change drastically Otherwise, we should not use this policy. or can we use it for all cases? I think It should be documented in the proposal. I have some questions - do we need to consider the

??????[Discuss][PIP-164] Support split bundle by flow or qps

2022-08-22 Thread lordcheng10
The implementation logic has been modified, and the corresponding example has also been modified as follows: ## Motivation As we all know, Bundle split has 3 algorithms: - range_equally_divide - topic_count_equally_divide - specified_positions_divide However, none of these algorithms can

RE: 回复:[Discuss][PIP-164] Support split bundle by flow or qps

2022-07-28 Thread David Kjerrumgaard
IMHO a better solution would be to allow the bundles to be split into more units in a single operation. Instead of splitting one bundle into 2, split it into 4 or 8….. If we do that then we can make that configurable (with a reasonable default). On 2022/07/26 08:39:29 lordcheng10 wrote: >

Re:[Discuss][PIP-164] Support split bundle by flow or qps

2022-07-26 Thread lordcheng10
When the broker is just started, the traffic or QPS may not be stable at this time. At this time, we can solve the problem by adjusting the time interval of the first load reporting task. At present, the time interval for executing the report task is 5 seconds, and it is not configurable. I can

[Discuss][PIP-164] Support split bundle by flow or qps

2022-07-26 Thread lordcheng10
Hi Pulsar Community, This is a PIP discussion on how to support splitting bundles according to flow or qps. The issue can be found: https://github.com/apache/pulsar/issues/16782 I copy the content here for convenience, any suggestions are welcome and appreciated. ## Motivation As we

??????[Discuss][PIP-164] Support split bundle by flow or qps

2022-07-26 Thread lordcheng10
When the broker is just started, the traffic or QPS may not be stable at this time. At this time, we can solve the problem by adjusting the time interval of the first load reporting task. The time interval for executing the report task is 5 seconds, and it is not configurable. I can add a

??????[Discuss][PIP-164] Support split bundle by flow or qps

2022-07-25 Thread lordcheng10
1. Similar problems also exist in topic_count_equally_divide. In some scenarios, topics are also temporary, because they will be created and destroyed continuously. 2. The traffic and qps of most scenarios are relatively stable. For scenarios where traffic and qps change greatly and frequently,

Re: [Discuss][PIP-164] Support split bundle by flow or qps

2022-07-25 Thread Haiting Jiang
Hi, This is an interesting proposal and sure it can solve quite some load balance case. My concern on this is that, relative speaking, the QPS and flow data is temporary, it may changes a lot, so it's easy to create some bad cases, especially on startup. Any idea on this matter? Thanks,

??????[Discuss][PIP-164] Support split bundle by flow or qps

2022-07-25 Thread lordcheng10
The PIP number is duplicated, the updated PIP number is PIP-169.

[Discuss][PIP-164] Support split bundle by flow or qps

2022-07-25 Thread lordcheng10
Hi Pulsar Community, This is a PIP discussion on how to support splitting bundles according to flow or qps. The issue can be found:https://github.com/apache/pulsar/issues/16782 I copy the content here for convenience, any suggestions are welcome and appreciated. ## Motivation As we all