https://issues.apache.org/bugzilla/show_bug.cgi?id=55897
Bug ID: 55897
Summary: [PATCH]prefork_mpm patch with SO_REUSEPORT support
Product: Apache httpd-2
Version: 2.4.7
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: mpm_prefork
Assignee: [email protected]
Reporter: [email protected]
Created attachment 31124
--> https://issues.apache.org/bugzilla/attachment.cgi?id=31124&action=edit
[PATCH]prefork_mpm patch with SO_REUSEPORT support
Issues we have found:
Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel Xeon
2600 series systems, using an open source three tier social networking web
server workload, revealed performance scaling issues. In current software
single listen statement (listen 80) provides better scalability due to
un-serialized accept. However, when system is under very high load, this can
lead to big number of child processes stuck in D state. On the other hand, the
serialized accept approach cannot scale with the high load either. In our
analysis, a 32-thread system, with 2 listen statements specified, could scale
to just 70% utilization, and a 64-thread system, with signal listen statement
specified (listen 80, 4 network interfaces), could scale to only 60%
utilization.
What we have changed:
Based on those findings, we created a prototype patch for prefork mpm which
extends performance and thread utilization. In Linux kernel newer than 3.9,
SO_REUSEPORT is enabled. This feature allows multiple sockets listen to the
same IP:port and automatically round robins connections. We use this feature to
create 4 duplicated listener records of the original one and partition the
child processes into 4 buckets. Each bucket listens to 1 IP:port. A mutex is
being used to guard only 1 child wakes up when there is a request comes in. In
case of old kernel which does not have the SO_REUSEPORT enabled, we modified
the "multiple listen statement case" by creating 1 listen record for each
listen statement and partitioning the child processes into different buckets.
Each bucket listens to 1 IP:port.
In the current work, we added the SO_REUSEPORT enablement into APR 1.5.0 and
filed a small patch with bugzilla ID 55894 (patch for SO_REUSEPORT) for this.
To review the patch, please use the APR patch as well.
Testing results:
Quick tests of the patch, running the same workload, demonstrated a 22%
throughput increase with 32-threads and 2 listen statements (Linux kernel
3.10.4). With the older kernel (Linux Kernel 3.8.8, without SO_REUSEPORT), 10%
performance gain was measured. With 64 threads, a 60% throughput increase was
achieved with just 1 listen statement (listen 80) and 1 active IP1 (Linux
Kernel 3.12.5). We also observed big reduction in response time, in addition to
the throughput improvement gained1 in our tests.
This is our first patch to the Apache community(besides the small APR change).
Please help us review it and let us know if there is anything we might revise
to improve it. Your feedback is very much appreciated.
Configuration:
<IfModule prefork.c>
ListenBacklog 105384
ServerLimit 105000
MaxClients 1024
MaxRequestsPerChild 0
StartServers 64
MinSpareServers 8
MaxSpareServers 16
</IfModule>
1. Software and workloads used in performance tests may have been optimized for
performance only on Intel microprocessors. Performance tests, such as SYSmark
and MobileMark, are measured using specific computer systems, components,
software, operations and functions. Any change to any of those factors may
cause the results to vary. You should consult other information and performance
tests to assist you in fully evaluating your contemplated purchases, including
the performance of that product when combined with other products.
Configurations: Xeon 2-socket 2680 server with 16x8GB DDR3-1333 and 4-socket
Xeon 4650 server with 48x4GB DDR3-1066 (running at 1.3GHz), shown with workload
as tested by Intel November 2013.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
