hpvd commented on issue #12800: URL: https://github.com/apache/pulsar/issues/12800#issuecomment-969295829
just a little translation to make this issue searchable in english language: Suspected pulsar-broker bug, but I can't confirm it. Contents. (1). Reasons for not using English (2). pulsar deployment version & details & architecture (3). Problems & Phenomena & Usage Statement & Desensitized Code 1.Problem 2. Phenomenon & usage statement & desensitized code (1). Reasons for not using English I am not good at English, although I can read it without any problem, but if I state the whole problem & phenomenon in English, I can't guarantee the accurate line of the statement, so I use Chinese. (2). pulsar deployment version & details & architecture pulsar version is 2.8.0, deployed on openjdk11, the specific version number is: 11.0.12. In aws overseas deployment, the model used is c5a.2xlarge (8c16g), a total of 3 units, each deployed a broker, bookie, zk. start command parameters are not modified are the default value. Deployment details and details. pulsar-7: deploying a production-grade 5-node pulsar cluster on aws https://mp.weixin.qq.com/s/YwCr-l2WcM4fJVg7NIx_HA (3). Problems & Phenomena & Usage Statement & Desensitization Code 1. Problem Individual partitions cannot be consumed by consumers under severe imbalance of topic messages in 40 partitions. The most recent one is that two partitions are stacked up to about 300,000 each (backlog value). The average size of this partition topic messages sent. pulsar_rate_in. pulsar_rate_out: The time of the spike is the time we found not consumed, but it is possible that there was already a problem before, so high is the restart of the consumer. 2. phenomenon and the use of statements & desensitization code The producer uses a batch send (average of about 8 is a batch, up to 40 is a batch), and is sent asynchronously, using key-sharding to different partitions of this topic, desensitization code. [CODE] consumer is using the key-sharding method of consumption, desensitization code [CODE] -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
