BewareMyPower commented on issue #229:
URL:
https://github.com/apache/pulsar-client-python/issues/229#issuecomment-2439835310
I rewrote the test script to avoid being affected by the pending queue
(because it would be blocked if the queue is full) and reduce the test time. I
tested various client versions with Python 3.8 on Ubuntu 20.04 WSL for 3 times
against Pulsar 4.0.0 standalone and the same topic locally.
```python
from pulsar import Client, CompressionType, Result
import os
import time
def send_callback(i, result, msg_id):
if result != Result.Ok:
print(f'{i} failed: {result}')
if __name__ == "__main__":
client = Client(service_url='pulsar://localhost:6650',
io_threads=4)
msg = os.urandom(100).hex().encode()
producer = client.create_producer(
'test-topic',
compression_type=CompressionType.LZ4,
batching_enabled=True,
batching_max_messages=1000, # batch size will be always 1000
batching_max_allowed_size_in_bytes=10485760,
batching_max_publish_delay_ms=10,
max_pending_messages=0, # avoid send_async being blocked due to full
queue
block_if_queue_full=True)
t1 = time.time()
for i in range(0, 200000):
producer.send_async(msg, lambda result, msg_id, i=i:
send_callback(i, result, msg_id))
producer.flush()
t2 = time.time()
print(f'send_async: {round(t2 - t1, 3)} s')
client.close()
```
| version | 1st | 2nd | 3rd |
| :- | -: | -: | -: |
| 2.10.2 | 7.169 | 7.211 | 7.083 |
| 3.1.0 | 6.768 | 6.788 | 6.943 |
| 3.2.0 | 7.445 | 7.336 | 7.427 |
| 3.3.0 | 7.451 | 7.454 | 7.435 |
| 3.4.0 | 8.955 | 9.369 | 8.256 |
| 3.5.0 | 9.676 | 9.653 | 10.097 |
P.S. 3.0.0 is not tested because it has a deadlock bug.
As we can see, actually 3.1.0 has better performance than 2.10.2. But there
are some significant performance regressions from 3.1.0 -> 3.2.0, 3.3.0 ->
3.4.0, 3.4.0 -> 3.5.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
