cwang9208 opened a new issue, #34478: URL: https://github.com/apache/arrow/issues/34478
### Describe the usage question you have. Please include as many useful details as possible. I'm benchmarking the performance of flight RPC using the built-in C++ benchmark server/client. Both the client and server run on the same machine. Below is the result for records_per_batch = 1000000. ``` $ ./arrow-flight-benchmark -records_per_stream 1000000000 -records_per_batch 1000000 -num_perf_runs 2 -num_streams 10 -num_threads 10 Testing method: DoGet Using spawned TCP server Server running with pid 3207456 Server host: localhost Server port: 31337 Number of perf runs: 2 Number of concurrent gets/puts: 10 Batch size: 32000000 Batches read: 20000 Bytes read: 640000000000 Nanos: 55198332155 Speed: 11057.4 MB/s Throughput: 362.33 batches/s Latency mean: 26931 us Latency quantile=0.5: 26270 us Latency quantile=0.95: 41160 us Latency quantile=0.99: 51025 us Latency max: 159917 us ``` And the result for records_per_batch = 5000000. ``` $ ./arrow-flight-benchmark -records_per_stream 1000000000 -records_per_batch 5000000 -num_perf_runs 2 -num_streams 10 -num_threads 10 Testing method: DoGet Using spawned TCP server Server running with pid 3206446 Server host: localhost Server port: 31337 Number of perf runs: 2 Number of concurrent gets/puts: 10 Batch size: 160000000 Batches read: 4000 Bytes read: 640000000000 Nanos: 79217518482 Speed: 7704.75 MB/s Throughput: 50.4939 batches/s Latency mean: 184366 us Latency quantile=0.5: 167930 us Latency quantile=0.95: 309036 us Latency quantile=0.99: 487428 us Latency max: 744113 us ``` I'm wondering why the Speed drops when the records_per_batch grows bigger? Can anyone help explain the reason behind this? ### Component(s) Benchmarking, C++, FlightRPC -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
