wolfstudy commented on issue #195: perf tools performance about 1/10 of java client URL: https://github.com/apache/pulsar-client-go/issues/195#issuecomment-599596820 In my local environment, the go client is indeed slower than the Java client, but there is no gap of 1/10. The test content is as follows: Pulsar Broker: 2.5.0 | Info | Value | |---| ---| |System| MacOS| |Disk| SSD| |Mem|16 GB 2400 MHz DDR4| |CPU|2.6 GHz Intel Core i7| #### Java Perf tool: ``` $ ./bin/pulsar-perf produce -r 100000 -s 10 testtopic 22:57:46.880 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Starting Pulsar perf producer with config: { "confFile" : "/Users/wolf4j/path/to/2.5.0/apache-pulsar-2.5.0/conf/client.conf", "topics" : [ "testtopic" ], "numTestThreads" : 1, "msgRate" : 100000, "msgSize" : 10, "numTopics" : 1, "numProducers" : 1, "serviceURL" : "pulsar://localhost:6650/", "authPluginClassName" : "", "authParams" : "", "maxOutstanding" : 1000, "maxPendingMessagesAcrossPartitions" : 50000, "maxConnections" : 100, "numMessages" : 0, "statsIntervalSeconds" : 0, "compression" : "NONE", "payloadFilename" : null, "payloadDelimiter" : "\\n", "batchTimeMillis" : 1.0, "batchMaxMessages" : 1000, "batchMaxBytes" : 4194304, "testTime" : 0, "warmupTimeSeconds" : 1.0, "tlsTrustCertsFilePath" : "", "encKeyName" : null, "encKeyFile" : null, "delay" : 0, "exitOnFailure" : false } 22:57:46.890 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Dumping latency stats to perf-producer-1584370666890.hgrm 22:57:46.890 [pulsar-perf-producer-exec-1-1] INFO org.apache.pulsar.testclient.PerformanceProducer - Started performance test thread 0 22:57:47.292 [pulsar-perf-producer-exec-1-1] INFO org.apache.pulsar.testclient.PerformanceProducer - Adding 1 publishers on topic testtopic 22:57:47.435 [pulsar-client-io-2-2] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xade1a46c, L:/127.0.0.1:53165 - R:localhost/127.0.0.1:6650]] Connected to server 22:57:47.504 [pulsar-client-io-2-5] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xb562db6a, L:/127.0.0.1:53166 - R:localhost/127.0.0.1:6650]] Connected to server 22:57:47.520 [pulsar-client-io-2-8] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x46ad80d3, L:/127.0.0.1:53167 - R:localhost/127.0.0.1:6650]] Connected to server 22:57:47.521 [pulsar-client-io-2-8] INFO org.apache.pulsar.client.impl.ClientCnx - [id: 0x46ad80d3, L:/127.0.0.1:53167 - R:localhost/127.0.0.1:6650] Connected through proxy to target broker at 127.0.0.1:6650 22:57:47.525 [pulsar-client-io-2-8] INFO org.apache.pulsar.client.impl.ProducerImpl - [testtopic] [null] Creating producer on cnx [id: 0x46ad80d3, L:/127.0.0.1:53167 - R:localhost/127.0.0.1:6650] 22:57:47.563 [pulsar-client-io-2-8] INFO org.apache.pulsar.client.impl.ProducerImpl - [testtopic] [standalone-0-3] Created producer on cnx [id: 0x46ad80d3, L:/127.0.0.1:53167 - R:localhost/127.0.0.1:6650] 22:57:47.575 [pulsar-perf-producer-exec-1-1] INFO org.apache.pulsar.testclient.PerformanceProducer - Created 1 producers 22:57:47.603 [pulsar-timer-5-1] WARN com.scurrilous.circe.checksum.Crc32cIntChecksum - Failed to load Circe JNI library. Falling back to Java based CRC32c provider 22:57:56.914 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 93080.3 msg/s --- 7.1 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.555 ms - med: 3.325 - 95pct: 4.543 - 99pct: 4.826 - 99.9pct: 8.015 - 99.99pct: 11.582 - Max: 11.688 22:58:06.967 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100000.7 msg/s --- 7.6 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.457 ms - med: 3.176 - 95pct: 4.414 - 99pct: 5.459 - 99.9pct: 12.570 - 99.99pct: 109.045 - Max: 109.091 22:58:16.988 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100008.4 msg/s --- 7.6 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.450 ms - med: 3.225 - 95pct: 4.613 - 99pct: 5.765 - 99.9pct: 7.215 - 99.99pct: 9.088 - Max: 9.140 22:58:27.006 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100005.5 msg/s --- 7.6 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.416 ms - med: 3.175 - 95pct: 4.434 - 99pct: 5.467 - 99.9pct: 7.650 - 99.99pct: 9.884 - Max: 10.064 22:58:37.023 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100002.5 msg/s --- 7.6 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.503 ms - med: 3.181 - 95pct: 4.443 - 99pct: 5.416 - 99.9pct: 15.427 - 99.99pct: 80.336 - Max: 80.492 22:58:47.039 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 99981.6 msg/s --- 7.6 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.420 ms - med: 3.152 - 95pct: 4.398 - 99pct: 5.355 - 99.9pct: 12.369 - 99.99pct: 17.801 - Max: 18.120 22:58:57.058 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 100021.9 msg/s --- 7.6 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.557 ms - med: 3.173 - 95pct: 4.516 - 99pct: 5.815 - 99.9pct: 17.282 - 99.99pct: 152.572 - Max: 152.746 22:59:07.075 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 99980.3 msg/s --- 7.6 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.395 ms - med: 3.130 - 95pct: 4.412 - 99pct: 5.582 - 99.9pct: 11.151 - 99.99pct: 19.556 - Max: 19.712 22:59:17.093 [main] INFO org.apache.pulsar.testclient.PerformanceProducer - Throughput produced: 99989.3 msg/s --- 7.6 Mbit/s --- failure 0.0 msg/s --- Latency: mean: 3.596 ms - med: 3.327 - 95pct: 5.207 - 99pct: 7.046 - 99.9pct: 12.245 - 99.99pct: 24.039 - Max: 24.192 ``` #### Go perf tool: ``` ./pulsar-perf produce -r 100000 -s 10 test-topic-1 INFO[23:09:02.306] Client config: { "ServiceURL": "pulsar://localhost:6650" } INFO[23:09:02.306] Producer config: { "Topic": "test-topic-1", "Rate": 100000, "BatchingTimeMillis": 1, "MessageSize": 10, "ProducerQueueSize": 1000 } INFO[23:09:02.306] Connecting to broker remote_addr="pulsar://localhost:6650" INFO[23:09:02.308] TCP connection established local_addr="127.0.0.1:53313" remote_addr="pulsar://localhost:6650" INFO[23:09:02.311] Connection is ready local_addr="127.0.0.1:53313" remote_addr="pulsar://localhost:6650" INFO[23:09:02.322] Connecting to broker remote_addr="pulsar://localhost:6650" INFO[23:09:02.323] TCP connection established local_addr="127.0.0.1:53315" remote_addr="pulsar://localhost:6650" INFO[23:09:02.324] Connection is ready local_addr="127.0.0.1:53315" remote_addr="pulsar://localhost:6650" INFO[23:09:02.596] Created producer cnx="127.0.0.1:53315 -> 127.0.0.1:6650" producer_name=standalone-0-5 topic="persistent://public/default/test-topic-1" INFO[23:09:12.596] Stats - Publish rate: 52444.1 msg/s - 4.0 Mbps - Latency ms: 50% 17.8 -95% 28.6 - 99% 55.3 - 99.9% 55.3 - max 55.3 INFO[23:09:22.597] Stats - Publish rate: 45959.5 msg/s - 3.5 Mbps - Latency ms: 50% 23.0 -95% 28.0 - 99% 62.9 - 99.9% 62.9 - max 62.9 INFO[23:09:32.597] Stats - Publish rate: 43857.1 msg/s - 3.3 Mbps - Latency ms: 50% 24.6 -95% 27.9 - 99% 32.5 - 99.9% 73.7 - max 73.7 INFO[23:09:42.597] Stats - Publish rate: 41038.3 msg/s - 3.1 Mbps - Latency ms: 50% 22.1 -95% 36.8 - 99% 134.3 - 99.9% 134.3 - max 134.3 INFO[23:09:52.597] Stats - Publish rate: 42440.9 msg/s - 3.2 Mbps - Latency ms: 50% 20.5 -95% 41.3 - 99% 70.4 - 99.9% 70.4 - max 70.4 INFO[23:10:02.598] Stats - Publish rate: 44944.6 msg/s - 3.4 Mbps - Latency ms: 50% 22.2 -95% 29.5 - 99% 63.8 - 99.9% 63.8 - max 63.8 INFO[23:10:12.598] Stats - Publish rate: 39560.0 msg/s - 3.0 Mbps - Latency ms: 50% 27.2 -95% 31.9 - 99% 49.3 - 99.9% 49.3 - max 49.3 INFO[23:10:22.598] Stats - Publish rate: 39589.7 msg/s - 3.0 Mbps - Latency ms: 50% 25.8 -95% 36.0 - 99% 85.7 - 99.9% 85.7 - max 85.7 INFO[23:10:32.599] Stats - Publish rate: 39521.6 msg/s - 3.0 Mbps - Latency ms: 50% 26.3 -95% 34.2 - 99% 57.5 - 99.9% 57.5 - max 57.5 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
