lzxuan opened a new issue, #7244: URL: https://github.com/apache/apisix/issues/7244
### Description Following [this Slack thread](https://the-asf.slack.com/archives/CUC5MN17A/p1653900217440819), where I benchmarked APISIX running on a single-node K3s of version `v1.23.6+k3s1` atop `Ubuntu 20.04` VM with `8-core Intel Xeon Processor (Skylake, IBRS)` CPU and `8192MB` memory, the QPS obtained was way below than what was stated in [the official benchmark page](https://apisix.apache.org/docs/apisix/benchmark/#benchmark-test-for-reverse-proxy-enabled-2-plugins). The benchmark result was obtained using `wrk -d 60 --latency` (following [the official benchmark page](https://apisix.apache.org/docs/apisix/benchmark/#benchmark-test-for-reverse-proxy-enabled-2-plugins)): OpenResty upstream with 1KB response: | Number of CPU Core | 1 | |--------------------|:--------:| | Average Latency | 178.18us | | Average Req/Sec | 30.70k | With APISIX (without any plugin enabled): | Number of CPU Core | 1 | 2 | 4 | |--------------------|:------:|:--------:|:--------:| | Average Latency | 0.94ms | 559.90us | 391.50us | | Average Req/Sec | 5.49k | 9.36k | 14.62k | With APISIX (with only prometheus plugin enabled): | Number of CPU Core | 1 | 2 | 4 | |--------------------|:------:|:--------:|:--------:| | Average Latency | 1.19ms | 678.52us | 460.13us | | Average Req/Sec | 4.29k | 7.74k | 12.58k | However, I was later being pointed out that the wrk command should be `wrk -d 5 -c 16` (following [the official benchmark script](https://github.com/apache/apisix/blob/master/benchmark/run.sh)). I then reran my benchmark on n1-highcpu-8 (8 vCPUs, 7.2 GB memory) on Google Cloud, with APISIX still running in a K3s cluster. The benchmark result of APISIX with `worker_processes` set to 1: ``` $ wrk -d 5 -c 16 http://up.test.com:30080 Running 5s test @ http://up.test.com:30080 2 threads and 16 connections Thread Stats Avg Stdev Max +/- Stdev Latency 1.72ms 1.05ms 32.26ms 99.08% Req/Sec 4.79k 497.22 5.71k 70.00% 47637 requests in 5.00s, 8.72MB read Requests/sec: 9524.89 Transfer/sec: 1.74MB ``` The QPS did increase but it is highly suspected to be solely due to the change of the wrk command to using more connections, as I obtained similar figures when running the second wrk command on the first Ubuntu VM. Nonetheless, the new QPS obtained is still way below the 15k QPS stated in the official benchmark page. **Steps to Reproduce:** 1. Create `n1-highcpu-8` compute instance and SSH to it. 2. Git clone and helm install (apisix)[https://github.com/lzxuan/apisix-prometheus-helm-chart]. 3. Kubectl apply (upstream)[https://gist.github.com/lzxuan/f7dbc9e9a7c30ed273463e1011b77e99]. 4. `sudo echo '127.0.0.1 up.test.com' >> /etc/hosts`. 5. Add APISIX route to upstream with host set to `up.test.com`. 6. Install wrk and `wrk -d 5 -c 16 http://up.test.com:30080`. ### Environment - ASISIX Helm chart version: `0.10.0` - APISIX version: `2.14.1` - Operating system: `Linux instance-1 5.13.0-1031-gcp #37~20.04.1-Ubuntu SMP Thu Jun 9 13:25:43 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux` - OpenResty version: `1.19.9.1` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
