RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Jim,

I found that original patch code does not guard "ap_daemons_to_start" to be 
equal or bigger than num_buckets. Since each bucket has a dedicated listener 
assigned to it, I think we need to make sure that each bucket has at least 1 
child process to start with.

I created this small patch to fix the issue. Can you please help to add the 
changes into the trunk?

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi [mailto:yingqi...@intel.com] 
Sent: Thursday, June 05, 2014 9:12 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Thanks very much for your help!

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Thursday, June 05, 2014 6:38 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Committed r1600656

Thx
On Jun 4, 2014, at 3:39 PM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> I just found that prefork and worker has issue with restart. Event mpm code 
> is good. 
> 
> I created this small patch to fix the issue for both prefork and worker. The 
> patch is based on rev #1600451.
> 
> Can you please help add the changes in the trunk?
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Tuesday, June 03, 2014 8:50 AM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
> 
> Thank you very much for your help!
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com]
> Sent: Tuesday, June 03, 2014 8:31 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
> 
> Next on the agenda is to push into eventopt
> 
> On Jun 3, 2014, at 10:21 AM, Jim Jagielski  wrote:
> 
>> FTR: I saw no reason to try to handle both patches... I used the 
>> so_reuseport patch as the primary patch to focus on.
>> 
>> I have some minor changes coming up to follow-up post the initial 
>> commit
>> 
>> On Jun 3, 2014, at 8:51 AM, Jim Jagielski  wrote:
>> 
>>> I have folded this into trunk and am currently fixing some compile 
>>> warnings and errors...
>>> 
>>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:
>>> 
>>>> Hi Jim,
>>>> 
>>>> Personally, I think the second approach is better, it keeps 
>>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original 
>>>> ones, only modifies dummy_connection (). Please let me know if you have 
>>>> different opinions.
>>>> 
>>>> Attached is the latest version of the two patches. They were both 
>>>> generated against trunk rev. 1598561. Please review them and let me know 
>>>> if there is anything missing.
>>>> 
>>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>>> Sent: Saturday, May 31, 2014 11:48 PM
>>>> To: dev@httpd.apache.org
>>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>>> SO_REUSEPORT support
>>>> 
>>>> Hi Jim,
>>>> 
>>>> Regarding to your comment #2, yes, you are right, it should be 
>>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much 
>>>> for catching this.
>>>> 
>>>> Regarding to your comment #1, the patch modifies the 
>>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
>>>> child_bucket). Inside the function, the reference listen statement is 
>>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls 
>>>> dummy_connection(). 
>>>> 
>>>> Can we just modify the return of ap_mpm_pod_signal() from 
>>>> dummy_connection(pod) to dummy_connection(pod, 0) and add 
>>>> ap_mpm_pod_signal_ex()? 
>>>> 
>>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
>>>> modify dummy_connection() to send dummy data via all the duplicated listen 
>>>> statements. Then, we do not need child_bucket as the input parameter for 
>>>> dummy_connection(). In this case, we do not need adding 
>>>> ap_mpm_pod_signal_ex() too.
>>>> 
>>>> I already tested the code for the above approaches and they bot

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Thanks very much for your help!

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com] 
Sent: Thursday, June 05, 2014 6:38 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Committed r1600656

Thx
On Jun 4, 2014, at 3:39 PM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> I just found that prefork and worker has issue with restart. Event mpm code 
> is good. 
> 
> I created this small patch to fix the issue for both prefork and worker. The 
> patch is based on rev #1600451.
> 
> Can you please help add the changes in the trunk?
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Tuesday, June 03, 2014 8:50 AM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
> 
> Thank you very much for your help!
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com]
> Sent: Tuesday, June 03, 2014 8:31 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
> 
> Next on the agenda is to push into eventopt
> 
> On Jun 3, 2014, at 10:21 AM, Jim Jagielski  wrote:
> 
>> FTR: I saw no reason to try to handle both patches... I used the 
>> so_reuseport patch as the primary patch to focus on.
>> 
>> I have some minor changes coming up to follow-up post the initial 
>> commit
>> 
>> On Jun 3, 2014, at 8:51 AM, Jim Jagielski  wrote:
>> 
>>> I have folded this into trunk and am currently fixing some compile 
>>> warnings and errors...
>>> 
>>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:
>>> 
>>>> Hi Jim,
>>>> 
>>>> Personally, I think the second approach is better, it keeps 
>>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original 
>>>> ones, only modifies dummy_connection (). Please let me know if you have 
>>>> different opinions.
>>>> 
>>>> Attached is the latest version of the two patches. They were both 
>>>> generated against trunk rev. 1598561. Please review them and let me know 
>>>> if there is anything missing.
>>>> 
>>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>>> Sent: Saturday, May 31, 2014 11:48 PM
>>>> To: dev@httpd.apache.org
>>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>>> SO_REUSEPORT support
>>>> 
>>>> Hi Jim,
>>>> 
>>>> Regarding to your comment #2, yes, you are right, it should be 
>>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much 
>>>> for catching this.
>>>> 
>>>> Regarding to your comment #1, the patch modifies the 
>>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
>>>> child_bucket). Inside the function, the reference listen statement is 
>>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls 
>>>> dummy_connection(). 
>>>> 
>>>> Can we just modify the return of ap_mpm_pod_signal() from 
>>>> dummy_connection(pod) to dummy_connection(pod, 0) and add 
>>>> ap_mpm_pod_signal_ex()? 
>>>> 
>>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
>>>> modify dummy_connection() to send dummy data via all the duplicated listen 
>>>> statements. Then, we do not need child_bucket as the input parameter for 
>>>> dummy_connection(). In this case, we do not need adding 
>>>> ap_mpm_pod_signal_ex() too.
>>>> 
>>>> I already tested the code for the above approaches and they both work. 
>>>> 
>>>> Please let me know which way you think is better. I can quickly send you 
>>>> an update for review.
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Lu, Yingqi
>>>> Sent: Saturday, May 31, 2014 3:28 PM
>>>> To: 
>>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>>> SO_REUSEPORT support
>>>> 
>>>> Hi Jim,
>>>> 
>>>> Thanks very much for your email

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

Committed r1600656

Thx
On Jun 4, 2014, at 3:39 PM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> I just found that prefork and worker has issue with restart. Event mpm code 
> is good. 
> 
> I created this small patch to fix the issue for both prefork and worker. The 
> patch is based on rev #1600451.
> 
> Can you please help add the changes in the trunk?
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com] 
> Sent: Tuesday, June 03, 2014 8:50 AM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> Thank you very much for your help!
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com]
> Sent: Tuesday, June 03, 2014 8:31 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> Next on the agenda is to push into eventopt
> 
> On Jun 3, 2014, at 10:21 AM, Jim Jagielski  wrote:
> 
>> FTR: I saw no reason to try to handle both patches... I used the 
>> so_reuseport patch as the primary patch to focus on.
>> 
>> I have some minor changes coming up to follow-up post the initial 
>> commit
>> 
>> On Jun 3, 2014, at 8:51 AM, Jim Jagielski  wrote:
>> 
>>> I have folded this into trunk and am currently fixing some compile 
>>> warnings and errors...
>>> 
>>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:
>>> 
>>>> Hi Jim,
>>>> 
>>>> Personally, I think the second approach is better, it keeps 
>>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original 
>>>> ones, only modifies dummy_connection (). Please let me know if you have 
>>>> different opinions.
>>>> 
>>>> Attached is the latest version of the two patches. They were both 
>>>> generated against trunk rev. 1598561. Please review them and let me know 
>>>> if there is anything missing.
>>>> 
>>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>>> Sent: Saturday, May 31, 2014 11:48 PM
>>>> To: dev@httpd.apache.org
>>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>>> SO_REUSEPORT support
>>>> 
>>>> Hi Jim,
>>>> 
>>>> Regarding to your comment #2, yes, you are right, it should be 
>>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much 
>>>> for catching this.
>>>> 
>>>> Regarding to your comment #1, the patch modifies the 
>>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
>>>> child_bucket). Inside the function, the reference listen statement is 
>>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls 
>>>> dummy_connection(). 
>>>> 
>>>> Can we just modify the return of ap_mpm_pod_signal() from 
>>>> dummy_connection(pod) to dummy_connection(pod, 0) and add 
>>>> ap_mpm_pod_signal_ex()? 
>>>> 
>>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
>>>> modify dummy_connection() to send dummy data via all the duplicated listen 
>>>> statements. Then, we do not need child_bucket as the input parameter for 
>>>> dummy_connection(). In this case, we do not need adding 
>>>> ap_mpm_pod_signal_ex() too.
>>>> 
>>>> I already tested the code for the above approaches and they both work. 
>>>> 
>>>> Please let me know which way you think is better. I can quickly send you 
>>>> an update for review.
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Lu, Yingqi
>>>> Sent: Saturday, May 31, 2014 3:28 PM
>>>> To: 
>>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>>> SO_REUSEPORT support
>>>> 
>>>> Hi Jim,
>>>> 
>>>> Thanks very much for your email! I will look into both of them and send an 
>>>> update tonight!
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrot

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Jim,

I just found that prefork and worker has issue with restart. Event mpm code is 
good. 

I created this small patch to fix the issue for both prefork and worker. The 
patch is based on rev #1600451.

Can you please help add the changes in the trunk?

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi [mailto:yingqi...@intel.com] 
Sent: Tuesday, June 03, 2014 8:50 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Thank you very much for your help!

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Tuesday, June 03, 2014 8:31 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Next on the agenda is to push into eventopt

On Jun 3, 2014, at 10:21 AM, Jim Jagielski  wrote:

> FTR: I saw no reason to try to handle both patches... I used the 
> so_reuseport patch as the primary patch to focus on.
> 
> I have some minor changes coming up to follow-up post the initial 
> commit
> 
> On Jun 3, 2014, at 8:51 AM, Jim Jagielski  wrote:
> 
>> I have folded this into trunk and am currently fixing some compile 
>> warnings and errors...
>> 
>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:
>> 
>>> Hi Jim,
>>> 
>>> Personally, I think the second approach is better, it keeps 
>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones, 
>>> only modifies dummy_connection (). Please let me know if you have different 
>>> opinions.
>>> 
>>> Attached is the latest version of the two patches. They were both generated 
>>> against trunk rev. 1598561. Please review them and let me know if there is 
>>> anything missing.
>>> 
>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -----Original Message-
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>> Sent: Saturday, May 31, 2014 11:48 PM
>>> To: dev@httpd.apache.org
>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>> SO_REUSEPORT support
>>> 
>>> Hi Jim,
>>> 
>>> Regarding to your comment #2, yes, you are right, it should be 
>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much 
>>> for catching this.
>>> 
>>> Regarding to your comment #1, the patch modifies the 
>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
>>> child_bucket). Inside the function, the reference listen statement is 
>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls 
>>> dummy_connection(). 
>>> 
>>> Can we just modify the return of ap_mpm_pod_signal() from 
>>> dummy_connection(pod) to dummy_connection(pod, 0) and add 
>>> ap_mpm_pod_signal_ex()? 
>>> 
>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
>>> modify dummy_connection() to send dummy data via all the duplicated listen 
>>> statements. Then, we do not need child_bucket as the input parameter for 
>>> dummy_connection(). In this case, we do not need adding 
>>> ap_mpm_pod_signal_ex() too.
>>> 
>>> I already tested the code for the above approaches and they both work. 
>>> 
>>> Please let me know which way you think is better. I can quickly send you an 
>>> update for review.
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -Original Message-
>>> From: Lu, Yingqi
>>> Sent: Saturday, May 31, 2014 3:28 PM
>>> To: 
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>> SO_REUSEPORT support
>>> 
>>> Hi Jim,
>>> 
>>> Thanks very much for your email! I will look into both of them and send an 
>>> update tonight!
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
>>>> 
>>>> I also see:
>>>> 
>>>>  /* kill off the idle ones */
>>>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>>>> +for (i = 0; i < num_buckets; i++) {
>>>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>>>> +}
>>>> 
>>>> 
>>>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
>>>> retained->max_daemons_limit, i); ??
>>>> 
>>>> /**
&g

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Thank you very much for your help!

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com] 
Sent: Tuesday, June 03, 2014 8:31 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Next on the agenda is to push into eventopt

On Jun 3, 2014, at 10:21 AM, Jim Jagielski  wrote:

> FTR: I saw no reason to try to handle both patches... I used the 
> so_reuseport patch as the primary patch to focus on.
> 
> I have some minor changes coming up to follow-up post the initial 
> commit
> 
> On Jun 3, 2014, at 8:51 AM, Jim Jagielski  wrote:
> 
>> I have folded this into trunk and am currently fixing some compile 
>> warnings and errors...
>> 
>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:
>> 
>>> Hi Jim,
>>> 
>>> Personally, I think the second approach is better, it keeps 
>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones, 
>>> only modifies dummy_connection (). Please let me know if you have different 
>>> opinions.
>>> 
>>> Attached is the latest version of the two patches. They were both generated 
>>> against trunk rev. 1598561. Please review them and let me know if there is 
>>> anything missing.
>>> 
>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -----Original Message-
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>> Sent: Saturday, May 31, 2014 11:48 PM
>>> To: dev@httpd.apache.org
>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>> SO_REUSEPORT support
>>> 
>>> Hi Jim,
>>> 
>>> Regarding to your comment #2, yes, you are right, it should be 
>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much 
>>> for catching this.
>>> 
>>> Regarding to your comment #1, the patch modifies the 
>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
>>> child_bucket). Inside the function, the reference listen statement is 
>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls 
>>> dummy_connection(). 
>>> 
>>> Can we just modify the return of ap_mpm_pod_signal() from 
>>> dummy_connection(pod) to dummy_connection(pod, 0) and add 
>>> ap_mpm_pod_signal_ex()? 
>>> 
>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
>>> modify dummy_connection() to send dummy data via all the duplicated listen 
>>> statements. Then, we do not need child_bucket as the input parameter for 
>>> dummy_connection(). In this case, we do not need adding 
>>> ap_mpm_pod_signal_ex() too.
>>> 
>>> I already tested the code for the above approaches and they both work. 
>>> 
>>> Please let me know which way you think is better. I can quickly send you an 
>>> update for review.
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -Original Message-
>>> From: Lu, Yingqi
>>> Sent: Saturday, May 31, 2014 3:28 PM
>>> To: 
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>> SO_REUSEPORT support
>>> 
>>> Hi Jim,
>>> 
>>> Thanks very much for your email! I will look into both of them and send an 
>>> update tonight!
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
>>>> 
>>>> I also see:
>>>> 
>>>>  /* kill off the idle ones */
>>>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>>>> +for (i = 0; i < num_buckets; i++) {
>>>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>>>> +}
>>>> 
>>>> 
>>>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
>>>> retained->max_daemons_limit, i); ??
>>>> 
>>>> /**
>>>> * Write data to the pipe-of-death, signalling that all child 
>>>> process
>>>> * should die.
>>>> * @param pod The pipe-of-death to write to.
>>>> * @param num The number of child processes to kill
>>>> + * @param my_bucket the bucket that holds the dying child process.
>>>> */
>>>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>>>> +AP_DECLARE(void) ap_mpm_pod_killpg(

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

Next on the agenda is to push into eventopt

On Jun 3, 2014, at 10:21 AM, Jim Jagielski  wrote:

> FTR: I saw no reason to try to handle both patches... I used
> the so_reuseport patch as the primary patch to focus on.
> 
> I have some minor changes coming up to follow-up post
> the initial commit
> 
> On Jun 3, 2014, at 8:51 AM, Jim Jagielski  wrote:
> 
>> I have folded this into trunk and am currently fixing some
>> compile warnings and errors...
>> 
>> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:
>> 
>>> Hi Jim,
>>> 
>>> Personally, I think the second approach is better, it keeps 
>>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones, 
>>> only modifies dummy_connection (). Please let me know if you have different 
>>> opinions.
>>> 
>>> Attached is the latest version of the two patches. They were both generated 
>>> against trunk rev. 1598561. Please review them and let me know if there is 
>>> anything missing.
>>> 
>>> I already updated the Bugzilla database for the item 55897 and item 56279.
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -----Original Message-
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com] 
>>> Sent: Saturday, May 31, 2014 11:48 PM
>>> To: dev@httpd.apache.org
>>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
>>> support
>>> 
>>> Hi Jim,
>>> 
>>> Regarding to your comment #2, yes, you are right, it should be 
>>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much 
>>> for catching this.
>>> 
>>> Regarding to your comment #1, the patch modifies the 
>>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
>>> child_bucket). Inside the function, the reference listen statement is 
>>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls 
>>> dummy_connection(). 
>>> 
>>> Can we just modify the return of ap_mpm_pod_signal() from 
>>> dummy_connection(pod) to dummy_connection(pod, 0) and add 
>>> ap_mpm_pod_signal_ex()? 
>>> 
>>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
>>> modify dummy_connection() to send dummy data via all the duplicated listen 
>>> statements. Then, we do not need child_bucket as the input parameter for 
>>> dummy_connection(). In this case, we do not need adding 
>>> ap_mpm_pod_signal_ex() too.
>>> 
>>> I already tested the code for the above approaches and they both work. 
>>> 
>>> Please let me know which way you think is better. I can quickly send you an 
>>> update for review.
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -Original Message-
>>> From: Lu, Yingqi
>>> Sent: Saturday, May 31, 2014 3:28 PM
>>> To: 
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
>>> support
>>> 
>>> Hi Jim,
>>> 
>>> Thanks very much for your email! I will look into both of them and send an 
>>> update tonight!
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
>>>> 
>>>> I also see:
>>>> 
>>>>  /* kill off the idle ones */
>>>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>>>> +for (i = 0; i < num_buckets; i++) {
>>>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>>>> +}
>>>> 
>>>> 
>>>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
>>>> retained->max_daemons_limit, i); ??
>>>> 
>>>> /**
>>>> * Write data to the pipe-of-death, signalling that all child process
>>>> * should die.
>>>> * @param pod The pipe-of-death to write to.
>>>> * @param num The number of child processes to kill
>>>> + * @param my_bucket the bucket that holds the dying child process.
>>>> */
>>>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>>>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int 
>>>> +child_bucket);
>>>> 
>>>> Isn't 'num' the same in both implementation??
>>>> 
>>>>> On May 31, 2014, at 12:03 PM, Jim Jagielski  wrote:
>>>>> 
>>>>> Sor

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

FTR: I saw no reason to try to handle both patches... I used
the so_reuseport patch as the primary patch to focus on.

I have some minor changes coming up to follow-up post
the initial commit

On Jun 3, 2014, at 8:51 AM, Jim Jagielski  wrote:

> I have folded this into trunk and am currently fixing some
> compile warnings and errors...
> 
> On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:
> 
>> Hi Jim,
>> 
>> Personally, I think the second approach is better, it keeps 
>> ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones, 
>> only modifies dummy_connection (). Please let me know if you have different 
>> opinions.
>> 
>> Attached is the latest version of the two patches. They were both generated 
>> against trunk rev. 1598561. Please review them and let me know if there is 
>> anything missing.
>> 
>> I already updated the Bugzilla database for the item 55897 and item 56279.
>> 
>> Thanks,
>> Yingqi
>> 
>> -Original Message-
>> From: Lu, Yingqi [mailto:yingqi...@intel.com] 
>> Sent: Saturday, May 31, 2014 11:48 PM
>> To: dev@httpd.apache.org
>> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
>> support
>> 
>> Hi Jim,
>> 
>> Regarding to your comment #2, yes, you are right, it should be 
>> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much for 
>> catching this.
>> 
>> Regarding to your comment #1, the patch modifies the 
>> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
>> child_bucket). Inside the function, the reference listen statement is 
>> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls dummy_connection(). 
>> 
>> Can we just modify the return of ap_mpm_pod_signal() from 
>> dummy_connection(pod) to dummy_connection(pod, 0) and add 
>> ap_mpm_pod_signal_ex()? 
>> 
>> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
>> modify dummy_connection() to send dummy data via all the duplicated listen 
>> statements. Then, we do not need child_bucket as the input parameter for 
>> dummy_connection(). In this case, we do not need adding 
>> ap_mpm_pod_signal_ex() too.
>> 
>> I already tested the code for the above approaches and they both work. 
>> 
>> Please let me know which way you think is better. I can quickly send you an 
>> update for review.
>> 
>> Thanks,
>> Yingqi
>> 
>> -Original Message-
>> From: Lu, Yingqi
>> Sent: Saturday, May 31, 2014 3:28 PM
>> To: 
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
>> support
>> 
>> Hi Jim,
>> 
>> Thanks very much for your email! I will look into both of them and send an 
>> update tonight!
>> 
>> Thanks,
>> Yingqi
>> 
>>> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
>>> 
>>> I also see:
>>> 
>>>   /* kill off the idle ones */
>>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>>> +for (i = 0; i < num_buckets; i++) {
>>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>>> +}
>>> 
>>> 
>>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
>>> retained->max_daemons_limit, i); ??
>>> 
>>> /**
>>> * Write data to the pipe-of-death, signalling that all child process
>>> * should die.
>>> * @param pod The pipe-of-death to write to.
>>> * @param num The number of child processes to kill
>>> + * @param my_bucket the bucket that holds the dying child process.
>>> */
>>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int 
>>> +child_bucket);
>>> 
>>> Isn't 'num' the same in both implementation??
>>> 
>>>> On May 31, 2014, at 12:03 PM, Jim Jagielski  wrote:
>>>> 
>>>> Sorry I didn't catch this earlier:
>>>> 
>>>> I see
>>>> 
>>>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491 
>>>> -0400
>>>> @@ -267,16 +267,18 @@
>>>> * Write data to the pipe-of-death, signalling that one child process
>>>> * should die.
>>>> * @param pod the pipe-of-death to write to.
>>>> + * @param my_bucket the bucket that holds the dying child process.
>>>> */
>>>> -A

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

I have folded this into trunk and am currently fixing some
compile warnings and errors...

On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> Personally, I think the second approach is better, it keeps ap_mpm_pod_signal 
> () and ap_mpm_pod_killpg () exactly as the original ones, only modifies 
> dummy_connection (). Please let me know if you have different opinions.
> 
> Attached is the latest version of the two patches. They were both generated 
> against trunk rev. 1598561. Please review them and let me know if there is 
> anything missing.
> 
> I already updated the Bugzilla database for the item 55897 and item 56279.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com] 
> Sent: Saturday, May 31, 2014 11:48 PM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> Hi Jim,
> 
> Regarding to your comment #2, yes, you are right, it should be 
> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much for 
> catching this.
> 
> Regarding to your comment #1, the patch modifies the 
> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
> child_bucket). Inside the function, the reference listen statement is 
> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls dummy_connection(). 
> 
> Can we just modify the return of ap_mpm_pod_signal() from 
> dummy_connection(pod) to dummy_connection(pod, 0) and add 
> ap_mpm_pod_signal_ex()? 
> 
> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
> modify dummy_connection() to send dummy data via all the duplicated listen 
> statements. Then, we do not need child_bucket as the input parameter for 
> dummy_connection(). In this case, we do not need adding 
> ap_mpm_pod_signal_ex() too.
> 
> I already tested the code for the above approaches and they both work. 
> 
> Please let me know which way you think is better. I can quickly send you an 
> update for review.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Lu, Yingqi
> Sent: Saturday, May 31, 2014 3:28 PM
> To: 
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> Hi Jim,
> 
> Thanks very much for your email! I will look into both of them and send an 
> update tonight!
> 
> Thanks,
> Yingqi
> 
>> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
>> 
>> I also see:
>> 
>>/* kill off the idle ones */
>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>> +for (i = 0; i < num_buckets; i++) {
>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>> +}
>> 
>> 
>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
>> retained->max_daemons_limit, i); ??
>> 
>> /**
>> * Write data to the pipe-of-death, signalling that all child process
>> * should die.
>> * @param pod The pipe-of-death to write to.
>> * @param num The number of child processes to kill
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int 
>> +child_bucket);
>> 
>> Isn't 'num' the same in both implementation??
>> 
>>> On May 31, 2014, at 12:03 PM, Jim Jagielski  wrote:
>>> 
>>> Sorry I didn't catch this earlier:
>>> 
>>> I see
>>> 
>>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491 
>>> -0400
>>> @@ -267,16 +267,18 @@
>>> * Write data to the pipe-of-death, signalling that one child process
>>> * should die.
>>> * @param pod the pipe-of-death to write to.
>>> + * @param my_bucket the bucket that holds the dying child process.
>>> */
>>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int 
>>> +child_bucket);
>>> 
>>> We can change the API at this point. We could add another function, 
>>> eg ap_mpm_pod_signal_ex() which takes the int param, but we can't 
>>> modify ap_mpm_pod_signal() itself.
>>> 
>>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi  wrote:
>>>> 
>>>> Thank you very much!
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Jim Jagie

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Marion et Christophe JAILLET]

Hi,

 

see http://httpd.apache.org/lists.html#http-dev

You should just send a mail to  dev-unsubscr...@httpd.apache.org.

 

best regards,

 

CJ





> Message du 02/06/14 16:16
> De : "Mihai Iacob" 
> A : dev@httpd.apache.org
> Copie à : 
> Objet : Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
>
> How do I unsubscribe from this list ?
> 
> Regards,
> Mihai Iacob
> DB2 Security Development
> DB2 pureScale Development
> Phone: (905) 413-5378
> Email: mia...@ca.ibm.com
>

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Mihai Iacob]

How do I unsubscribe from this list ?

Regards,
Mihai Iacob
DB2 Security Development
DB2 pureScale Development
Phone: (905) 413-5378
Email: mia...@ca.ibm.com



From:   Jim Jagielski 
To: dev@httpd.apache.org
Date:   06/02/2014 07:36 AM
Subject:Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
SO_REUSEPORT support



Thanks!
On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:

> Hi Jim,
>
> Personally, I think the second approach is better, it keeps
ap_mpm_pod_signal () and ap_mpm_pod_killpg () exactly as the original ones,
only modifies dummy_connection (). Please let me know if you have different
opinions.
>
> Attached is the latest version of the two patches. They were both
generated against trunk rev. 1598561. Please review them and let me know if
there is anything missing.
>
> I already updated the Bugzilla database for the item 55897 and item
56279.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Saturday, May 31, 2014 11:48 PM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
SO_REUSEPORT support
>
> Hi Jim,
>
> Regarding to your comment #2, yes, you are right, it should be
ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much
for catching this.
>
> Regarding to your comment #1, the patch modifies the dummy_connection
(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int child_bucket).
Inside the function, the reference listen statement is mpm_listen
[child_bucket]. And, ap_mpm_pod_signal() calls dummy_connection().
>
> Can we just modify the return of ap_mpm_pod_signal() from
dummy_connection(pod) to dummy_connection(pod, 0) and add
ap_mpm_pod_signal_ex()?
>
> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can
modify dummy_connection() to send dummy data via all the duplicated listen
statements. Then, we do not need child_bucket as the input parameter for
dummy_connection(). In this case, we do not need adding
ap_mpm_pod_signal_ex() too.
>
> I already tested the code for the above approaches and they both work.
>
> Please let me know which way you think is better. I can quickly send you
an update for review.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Lu, Yingqi
> Sent: Saturday, May 31, 2014 3:28 PM
> To: 
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
SO_REUSEPORT support
>
> Hi Jim,
>
> Thanks very much for your email! I will look into both of them and send
an update tonight!
>
> Thanks,
> Yingqi
>
>> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
>>
>> I also see:
>>
>>/* kill off the idle ones */
>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>> +for (i = 0; i < num_buckets; i++) {
>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>> +}
>>
>>
>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, retained->
max_daemons_limit, i); ??
>>
>> /**
>> * Write data to the pipe-of-death, signalling that all child process
>> * should die.
>> * @param pod The pipe-of-death to write to.
>> * @param num The number of child processes to kill
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int
>> +child_bucket);
>>
>> Isn't 'num' the same in both implementation??
>>
>>> On May 31, 2014, at 12:03 PM, Jim Jagielski  wrote:
>>>
>>> Sorry I didn't catch this earlier:
>>>
>>> I see
>>>
>>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16
13:07:03.892987491 -0400
>>> @@ -267,16 +267,18 @@
>>> * Write data to the pipe-of-death, signalling that one child process
>>> * should die.
>>> * @param pod the pipe-of-death to write to.
>>> + * @param my_bucket the bucket that holds the dying child process.
>>> */
>>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int
>>> +child_bucket);
>>>
>>> We can change the API at this point. We could add another function,
>>> eg ap_mpm_pod_signal_ex() which takes the int param, but we can't
>>> modify ap_mpm_pod_signal() itself.
>>>
>>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi  wrote:
>>>>
>>>> Thank you very much!
>>>>
>>>> Thanks,
>>>> Yingqi
>>>>
>>>

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

Thanks!
On Jun 2, 2014, at 4:22 AM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> Personally, I think the second approach is better, it keeps ap_mpm_pod_signal 
> () and ap_mpm_pod_killpg () exactly as the original ones, only modifies 
> dummy_connection (). Please let me know if you have different opinions.
> 
> Attached is the latest version of the two patches. They were both generated 
> against trunk rev. 1598561. Please review them and let me know if there is 
> anything missing.
> 
> I already updated the Bugzilla database for the item 55897 and item 56279.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Lu, Yingqi [mailto:yingqi...@intel.com] 
> Sent: Saturday, May 31, 2014 11:48 PM
> To: dev@httpd.apache.org
> Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> Hi Jim,
> 
> Regarding to your comment #2, yes, you are right, it should be 
> ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much for 
> catching this.
> 
> Regarding to your comment #1, the patch modifies the 
> dummy_connection(ap_pod_t *pod) to be dummy_connection(ap_pod_t *pod, int 
> child_bucket). Inside the function, the reference listen statement is 
> mpm_listen[child_bucket]. And, ap_mpm_pod_signal() calls dummy_connection(). 
> 
> Can we just modify the return of ap_mpm_pod_signal() from 
> dummy_connection(pod) to dummy_connection(pod, 0) and add 
> ap_mpm_pod_signal_ex()? 
> 
> Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
> modify dummy_connection() to send dummy data via all the duplicated listen 
> statements. Then, we do not need child_bucket as the input parameter for 
> dummy_connection(). In this case, we do not need adding 
> ap_mpm_pod_signal_ex() too.
> 
> I already tested the code for the above approaches and they both work. 
> 
> Please let me know which way you think is better. I can quickly send you an 
> update for review.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Lu, Yingqi
> Sent: Saturday, May 31, 2014 3:28 PM
> To: 
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> Hi Jim,
> 
> Thanks very much for your email! I will look into both of them and send an 
> update tonight!
> 
> Thanks,
> Yingqi
> 
>> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
>> 
>> I also see:
>> 
>>/* kill off the idle ones */
>> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
>> +for (i = 0; i < num_buckets; i++) {
>> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
>> +}
>> 
>> 
>> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
>> retained->max_daemons_limit, i); ??
>> 
>> /**
>> * Write data to the pipe-of-death, signalling that all child process
>> * should die.
>> * @param pod The pipe-of-death to write to.
>> * @param num The number of child processes to kill
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
>> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int 
>> +child_bucket);
>> 
>> Isn't 'num' the same in both implementation??
>> 
>>> On May 31, 2014, at 12:03 PM, Jim Jagielski  wrote:
>>> 
>>> Sorry I didn't catch this earlier:
>>> 
>>> I see
>>> 
>>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491 
>>> -0400
>>> @@ -267,16 +267,18 @@
>>> * Write data to the pipe-of-death, signalling that one child process
>>> * should die.
>>> * @param pod the pipe-of-death to write to.
>>> + * @param my_bucket the bucket that holds the dying child process.
>>> */
>>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int 
>>> +child_bucket);
>>> 
>>> We can change the API at this point. We could add another function, 
>>> eg ap_mpm_pod_signal_ex() which takes the int param, but we can't 
>>> modify ap_mpm_pod_signal() itself.
>>> 
>>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi  wrote:
>>>> 
>>>> Thank you very much!
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>> Sent: Friday, May 30, 2014 7:07 AM
>>>> To:

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Jim,

Regarding to your comment #2, yes, you are right, it should be 
ap_mpm_pod_killpg(pod, retained->max_daemons_limit, i). Thanks very much for 
catching this.

Regarding to your comment #1, the patch modifies the dummy_connection(ap_pod_t 
*pod) to be dummy_connection(ap_pod_t *pod, int child_bucket). Inside the 
function, the reference listen statement is mpm_listen[child_bucket]. And, 
ap_mpm_pod_signal() calls dummy_connection(). 

Can we just modify the return of ap_mpm_pod_signal() from dummy_connection(pod) 
to dummy_connection(pod, 0) and add ap_mpm_pod_signal_ex()? 

Or, if we need to keep ap_mpm_pod_signal() exactly as the original, I can 
modify dummy_connection() to send dummy data via all the duplicated listen 
statements. Then, we do not need child_bucket as the input parameter for 
dummy_connection(). In this case, we do not need adding ap_mpm_pod_signal_ex() 
too.

I already tested the code for the above approaches and they both work. 

Please let me know which way you think is better. I can quickly send you an 
update for review.

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi 
Sent: Saturday, May 31, 2014 3:28 PM
To: 
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Jim,

Thanks very much for your email! I will look into both of them and send an 
update tonight!

Thanks,
Yingqi

> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
> 
> I also see:
> 
> /* kill off the idle ones */
> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
> +for (i = 0; i < num_buckets; i++) {
> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
> +}
> 
> 
> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
> retained->max_daemons_limit, i); ??
> 
> /**
>  * Write data to the pipe-of-death, signalling that all child process
>  * should die.
>  * @param pod The pipe-of-death to write to.
>  * @param num The number of child processes to kill
> + * @param my_bucket the bucket that holds the dying child process.
>  */
> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int 
> +child_bucket);
> 
> Isn't 'num' the same in both implementation??
> 
>> On May 31, 2014, at 12:03 PM, Jim Jagielski  wrote:
>> 
>> Sorry I didn't catch this earlier:
>> 
>> I see
>> 
>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491 
>> -0400
>> @@ -267,16 +267,18 @@
>> * Write data to the pipe-of-death, signalling that one child process
>> * should die.
>> * @param pod the pipe-of-death to write to.
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int 
>> +child_bucket);
>> 
>> We can change the API at this point. We could add another function, 
>> eg ap_mpm_pod_signal_ex() which takes the int param, but we can't 
>> modify ap_mpm_pod_signal() itself.
>> 
>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi  wrote:
>>> 
>>> Thank you very much!
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -Original Message-
>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>> Sent: Friday, May 30, 2014 7:07 AM
>>> To: dev@httpd.apache.org
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>> SO_REUSEPORT support
>>> 
>>> Thx! Let me review. My plan is to fold into trunk this weekend.
>>> 
>>>> On May 16, 2014, at 2:53 PM, Lu, Yingqi  wrote:
>>>> 
>>>> Hi Jim,
>>>> 
>>>> Thanks very much for clarifying this with me. I added #ifdef in the code 
>>>> to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does 
>>>> not use this parameter so that it remains the same.
>>>> 
>>>> Attached are the two most recent patches. I already updated the bugzilla 
>>>> #55897 as well.
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>> Sent: Thursday, May 15, 2014 7:53 AM
>>>> To: dev@httpd.apache.org
>>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>>> SO_REUSEPORT support
>>>> 
>>>> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
>>>> We could either check for that during config/

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Jim,

Thanks very much for your email! I will look into both of them and send an 
update tonight!

Thanks,
Yingqi

> On May 31, 2014, at 9:43 AM, "Jim Jagielski"  wrote:
> 
> I also see:
> 
> /* kill off the idle ones */
> -ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
> +for (i = 0; i < num_buckets; i++) {
> +ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
> +}
> 
> 
> Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
> retained->max_daemons_limit, i); ??
> 
> /**
>  * Write data to the pipe-of-death, signalling that all child process
>  * should die.
>  * @param pod The pipe-of-death to write to.
>  * @param num The number of child processes to kill
> + * @param my_bucket the bucket that holds the dying child process.
>  */
> -AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
> +AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int child_bucket);
> 
> Isn't 'num' the same in both implementation??
> 
>> On May 31, 2014, at 12:03 PM, Jim Jagielski  wrote:
>> 
>> Sorry I didn't catch this earlier:
>> 
>> I see
>> 
>> +++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491 
>> -0400
>> @@ -267,16 +267,18 @@
>> * Write data to the pipe-of-death, signalling that one child process
>> * should die.
>> * @param pod the pipe-of-death to write to.
>> + * @param my_bucket the bucket that holds the dying child process.
>> */
>> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
>> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int child_bucket);
>> 
>> We can change the API at this point. We could
>> add another function, eg ap_mpm_pod_signal_ex() which
>> takes the int param, but we can't modify ap_mpm_pod_signal()
>> itself.
>> 
>>> On May 30, 2014, at 11:15 AM, Lu, Yingqi  wrote:
>>> 
>>> Thank you very much!
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -Original Message-
>>> From: Jim Jagielski [mailto:j...@jagunet.com] 
>>> Sent: Friday, May 30, 2014 7:07 AM
>>> To: dev@httpd.apache.org
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
>>> support
>>> 
>>> Thx! Let me review. My plan is to fold into trunk this weekend.
>>> 
>>>> On May 16, 2014, at 2:53 PM, Lu, Yingqi  wrote:
>>>> 
>>>> Hi Jim,
>>>> 
>>>> Thanks very much for clarifying this with me. I added #ifdef in the code 
>>>> to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does 
>>>> not use this parameter so that it remains the same.
>>>> 
>>>> Attached are the two most recent patches. I already updated the bugzilla 
>>>> #55897 as well.
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>> Sent: Thursday, May 15, 2014 7:53 AM
>>>> To: dev@httpd.apache.org
>>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>>> SO_REUSEPORT support
>>>> 
>>>> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
>>>> We could either check for that during config/build or protect it with a 
>>>> #ifdef in the code (and create some logging so the admin nows if it was 
>>>> found or not).
>>>> 
>>>>> On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:
>>>>> 
>>>>> Hi Jim,
>>>>> 
>>>>> Thanks very much for your email.
>>>>> 
>>>>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside 
>>>>> listen.c file. If the feature is not supported on the OS (for example, 
>>>>> Linux kernel < 3.9), it will fall back to the original behavior. 
>>>>> 
>>>>> In the bucket patch, there is no need to check the params. With single 
>>>>> listen statement, it is just the default behavior. 
>>>>> 
>>>>> Please let me know if this answers your question.
>>>>> 
>>>>> Thanks,
>>>>> Yingqi
>>>>> 
>>>>> -Original Message-
>>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>>> Sent: Wednesday, May 14, 2014 6:57 AM
>>>>> To: dev@httpd.apache.org
>>>>> S

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

I also see:

 /* kill off the idle ones */
-ap_mpm_pod_killpg(pod, retained->max_daemons_limit);
+for (i = 0; i < num_buckets; i++) {
+ap_mpm_pod_killpg(pod[i], i, retained->max_daemons_limit);
+}


Is that right? Why isn't it: ap_mpm_pod_killpg(pod, 
retained->max_daemons_limit, i); ??

/**
  * Write data to the pipe-of-death, signalling that all child process
  * should die.
  * @param pod The pipe-of-death to write to.
  * @param num The number of child processes to kill
+ * @param my_bucket the bucket that holds the dying child process.
  */
-AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num);
+AP_DECLARE(void) ap_mpm_pod_killpg(ap_pod_t *pod, int num, int child_bucket);

Isn't 'num' the same in both implementation??

On May 31, 2014, at 12:03 PM, Jim Jagielski  wrote:

> Sorry I didn't catch this earlier:
> 
> I see
> 
> +++ httpd-trunk.new/include/mpm_common.h  2014-05-16 13:07:03.892987491 
> -0400
> @@ -267,16 +267,18 @@
>  * Write data to the pipe-of-death, signalling that one child process
>  * should die.
>  * @param pod the pipe-of-death to write to.
> + * @param my_bucket the bucket that holds the dying child process.
>  */
> -AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
> +AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int child_bucket);
> 
> We can change the API at this point. We could
> add another function, eg ap_mpm_pod_signal_ex() which
> takes the int param, but we can't modify ap_mpm_pod_signal()
> itself.
> 
> On May 30, 2014, at 11:15 AM, Lu, Yingqi  wrote:
> 
>> Thank you very much!
>> 
>> Thanks,
>> Yingqi
>> 
>> -Original Message-----
>> From: Jim Jagielski [mailto:j...@jagunet.com] 
>> Sent: Friday, May 30, 2014 7:07 AM
>> To: dev@httpd.apache.org
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
>> support
>> 
>> Thx! Let me review. My plan is to fold into trunk this weekend.
>> 
>> On May 16, 2014, at 2:53 PM, Lu, Yingqi  wrote:
>> 
>>> Hi Jim,
>>> 
>>> Thanks very much for clarifying this with me. I added #ifdef in the code to 
>>> check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not 
>>> use this parameter so that it remains the same.
>>> 
>>> Attached are the two most recent patches. I already updated the bugzilla 
>>> #55897 as well.
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -Original Message-
>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>> Sent: Thursday, May 15, 2014 7:53 AM
>>> To: dev@httpd.apache.org
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>> SO_REUSEPORT support
>>> 
>>> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
>>> We could either check for that during config/build or protect it with a 
>>> #ifdef in the code (and create some logging so the admin nows if it was 
>>> found or not).
>>> 
>>> On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:
>>> 
>>>> Hi Jim,
>>>> 
>>>> Thanks very much for your email.
>>>> 
>>>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
>>>> file. If the feature is not supported on the OS (for example, Linux kernel 
>>>> < 3.9), it will fall back to the original behavior. 
>>>> 
>>>> In the bucket patch, there is no need to check the params. With single 
>>>> listen statement, it is just the default behavior. 
>>>> 
>>>> Please let me know if this answers your question.
>>>> 
>>>> Thanks,
>>>> Yingqi
>>>> 
>>>> -Original Message-
>>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>>> Sent: Wednesday, May 14, 2014 6:57 AM
>>>> To: dev@httpd.apache.org
>>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>>> SO_REUSEPORT support
>>>> 
>>>> This is very cool!
>>>> 
>>>> mod_status assumes that sysconf() exists, but do we need to do a config 
>>>> check on the params we use in these patches?
>>>> We look OK on Linux, FreeBSD and OSX...
>>>> 
>>>> I'm +1 on folding into trunk.
>>>> 
>>>> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
>>>> 
>>>>> Dear All,
>>>>> 
>>>>> During the last couple weeks, I spent some time extending 

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

Sorry I didn't catch this earlier:

I see

+++ httpd-trunk.new/include/mpm_common.h2014-05-16 13:07:03.892987491 
-0400
@@ -267,16 +267,18 @@
  * Write data to the pipe-of-death, signalling that one child process
  * should die.
  * @param pod the pipe-of-death to write to.
+ * @param my_bucket the bucket that holds the dying child process.
  */
-AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod);
+AP_DECLARE(apr_status_t) ap_mpm_pod_signal(ap_pod_t *pod, int child_bucket);

We can change the API at this point. We could
add another function, eg ap_mpm_pod_signal_ex() which
takes the int param, but we can't modify ap_mpm_pod_signal()
itself.

On May 30, 2014, at 11:15 AM, Lu, Yingqi  wrote:

> Thank you very much!
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com] 
> Sent: Friday, May 30, 2014 7:07 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> Thx! Let me review. My plan is to fold into trunk this weekend.
> 
> On May 16, 2014, at 2:53 PM, Lu, Yingqi  wrote:
> 
>> Hi Jim,
>> 
>> Thanks very much for clarifying this with me. I added #ifdef in the code to 
>> check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not 
>> use this parameter so that it remains the same.
>> 
>> Attached are the two most recent patches. I already updated the bugzilla 
>> #55897 as well.
>> 
>> Thanks,
>> Yingqi
>> 
>> -Original Message-----
>> From: Jim Jagielski [mailto:j...@jagunet.com]
>> Sent: Thursday, May 15, 2014 7:53 AM
>> To: dev@httpd.apache.org
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>> SO_REUSEPORT support
>> 
>> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
>> We could either check for that during config/build or protect it with a 
>> #ifdef in the code (and create some logging so the admin nows if it was 
>> found or not).
>> 
>> On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:
>> 
>>> Hi Jim,
>>> 
>>> Thanks very much for your email.
>>> 
>>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
>>> file. If the feature is not supported on the OS (for example, Linux kernel 
>>> < 3.9), it will fall back to the original behavior. 
>>> 
>>> In the bucket patch, there is no need to check the params. With single 
>>> listen statement, it is just the default behavior. 
>>> 
>>> Please let me know if this answers your question.
>>> 
>>> Thanks,
>>> Yingqi
>>> 
>>> -Original Message-
>>> From: Jim Jagielski [mailto:j...@jagunet.com]
>>> Sent: Wednesday, May 14, 2014 6:57 AM
>>> To: dev@httpd.apache.org
>>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>> SO_REUSEPORT support
>>> 
>>> This is very cool!
>>> 
>>> mod_status assumes that sysconf() exists, but do we need to do a config 
>>> check on the params we use in these patches?
>>> We look OK on Linux, FreeBSD and OSX...
>>> 
>>> I'm +1 on folding into trunk.
>>> 
>>> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
>>> 
>>>> Dear All,
>>>> 
>>>> During the last couple weeks, I spent some time extending the original two 
>>>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
>>>> event). Attached is the latest version of the two patches. Bugzilla 
>>>> database has also been updated already. The ID for the two patches are 
>>>> #55897 and #56279. Please refer to messages below for details on both of 
>>>> the patches.
>>>> 
>>>> Quick test result on modern dual socket Intel platform (Linux Kernel
>>>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
>>>> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
>>>> listen statements: 2.33X throughput improvement
>>>> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 
>>>> listen statements: 35% throughput improvement
>>>> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 
>>>> listen statements: throughput parity, but 62% response time reduction 
>>>> (with patch, 38% response time as original SW)
>>>> 
>>>> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
>>>> 1.   Prefork MPM: 2 listen statem

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Thank you very much!

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com] 
Sent: Friday, May 30, 2014 7:07 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Thx! Let me review. My plan is to fold into trunk this weekend.

On May 16, 2014, at 2:53 PM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> Thanks very much for clarifying this with me. I added #ifdef in the code to 
> check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not 
> use this parameter so that it remains the same.
> 
> Attached are the two most recent patches. I already updated the bugzilla 
> #55897 as well.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com]
> Sent: Thursday, May 15, 2014 7:53 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
> 
> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
> We could either check for that during config/build or protect it with a 
> #ifdef in the code (and create some logging so the admin nows if it was found 
> or not).
> 
> On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:
> 
>> Hi Jim,
>> 
>> Thanks very much for your email.
>> 
>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
>> file. If the feature is not supported on the OS (for example, Linux kernel < 
>> 3.9), it will fall back to the original behavior. 
>> 
>> In the bucket patch, there is no need to check the params. With single 
>> listen statement, it is just the default behavior. 
>> 
>> Please let me know if this answers your question.
>> 
>> Thanks,
>> Yingqi
>> 
>> -Original Message-----
>> From: Jim Jagielski [mailto:j...@jagunet.com]
>> Sent: Wednesday, May 14, 2014 6:57 AM
>> To: dev@httpd.apache.org
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>> SO_REUSEPORT support
>> 
>> This is very cool!
>> 
>> mod_status assumes that sysconf() exists, but do we need to do a config 
>> check on the params we use in these patches?
>> We look OK on Linux, FreeBSD and OSX...
>> 
>> I'm +1 on folding into trunk.
>> 
>> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
>> 
>>> Dear All,
>>> 
>>> During the last couple weeks, I spent some time extending the original two 
>>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
>>> event). Attached is the latest version of the two patches. Bugzilla 
>>> database has also been updated already. The ID for the two patches are 
>>> #55897 and #56279. Please refer to messages below for details on both of 
>>> the patches.
>>> 
>>> Quick test result on modern dual socket Intel platform (Linux Kernel
>>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
>>> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
>>> listen statements: 2.33X throughput improvement
>>> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 
>>> listen statements: 35% throughput improvement
>>> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 
>>> listen statements: throughput parity, but 62% response time reduction (with 
>>> patch, 38% response time as original SW)
>>> 
>>> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
>>> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
>>> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>>> 
>>> In all the above testing cases, significant response time reductions are 
>>> observed, even with throughput improvements.
>>> 
>>> Please let me know your feedback and comments.
>>> 
>>> Thanks,
>>> Yingqi
>>> Software and workloads used in performance tests may have been optimized 
>>> for performance only on Intel microprocessors. Performance tests, such as 
>>> SYSmark and MobileMark, are measured using specific computer systems, 
>>> components, software, operations and functions. Any change to any of those 
>>> factors may cause the results to vary. You should consult other information 
>>> and performance tests to assist you in fully evaluating your contemplated 
>>> purchases, including the performance of that product when combined with 
>>> other products.
>>> 
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
&g

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Mihai Iacob]

how do I unsubsribe from this list ?

Regards,
Mihai Iacob
DB2 Security Development
DB2 pureScale Development
Phone: (905) 413-5378
Email: mia...@ca.ibm.com



From:   Jim Jagielski 
To: dev@httpd.apache.org
Date:   05/30/2014 10:07 AM
Subject:Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
SO_REUSEPORT support



Thx! Let me review. My plan is to fold into trunk
this weekend.

On May 16, 2014, at 2:53 PM, Lu, Yingqi  wrote:

> Hi Jim,
>
> Thanks very much for clarifying this with me. I added #ifdef in the code
to check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does
not use this parameter so that it remains the same.
>
> Attached are the two most recent patches. I already updated the bugzilla
#55897 as well.
>
> Thanks,
> Yingqi
>
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com]
> Sent: Thursday, May 15, 2014 7:53 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
SO_REUSEPORT support
>
> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
> We could either check for that during config/build or protect it with a
#ifdef in the code (and create some logging so the admin nows if it was
found or not).
>
> On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:
>
>> Hi Jim,
>>
>> Thanks very much for your email.
>>
>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside
listen.c file. If the feature is not supported on the OS (for example,
Linux kernel < 3.9), it will fall back to the original behavior.
>>
>> In the bucket patch, there is no need to check the params. With single
listen statement, it is just the default behavior.
>>
>> Please let me know if this answers your question.
>>
>> Thanks,
>> Yingqi
>>
>> -Original Message-----
>> From: Jim Jagielski [mailto:j...@jagunet.com]
>> Sent: Wednesday, May 14, 2014 6:57 AM
>> To: dev@httpd.apache.org
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>> SO_REUSEPORT support
>>
>> This is very cool!
>>
>> mod_status assumes that sysconf() exists, but do we need to do a config
check on the params we use in these patches?
>> We look OK on Linux, FreeBSD and OSX...
>>
>> I'm +1 on folding into trunk.
>>
>> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
>>
>>> Dear All,
>>>
>>> During the last couple weeks, I spent some time extending the original
two patches from prefork MPM only to all three Linux MPMs (prefork, worker
and event). Attached is the latest version of the two patches. Bugzilla
database has also been updated already. The ID for the two patches are
#55897 and #56279. Please refer to messages below for details on both of
the patches.
>>>
>>> Quick test result on modern dual socket Intel platform (Linux Kernel
>>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
>>> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement;
2 listen statements: 2.33X throughput improvement
>>> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2
listen statements: 35% throughput improvement
>>> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2
listen statements: throughput parity, but 62% response time reduction (with
patch, 38% response time as original SW)
>>>
>>> Bucket patch (bugzilla #56279, only impact multiple listen statement
case)
>>> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
>>> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>>>
>>> In all the above testing cases, significant response time reductions
are observed, even with throughput improvements.
>>>
>>> Please let me know your feedback and comments.
>>>
>>> Thanks,
>>> Yingqi
>>> Software and workloads used in performance tests may have been
optimized for performance only on Intel microprocessors. Performance tests,
such as SYSmark and MobileMark, are measured using specific computer
systems, components, software, operations and functions. Any change to any
of those factors may cause the results to vary. You should consult other
information and performance tests to assist you in fully evaluating your
contemplated purchases, including the performance of that product when
combined with other products.
>>>
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>> Sent: Monday, March 17, 2014 1:41 PM
>>> To: dev@httpd.apache.org
>>> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>>> SO_REUSEPORT support
>>>
>>> Dear all,
>>>
>>

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

Thx! Let me review. My plan is to fold into trunk
this weekend.

On May 16, 2014, at 2:53 PM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> Thanks very much for clarifying this with me. I added #ifdef in the code to 
> check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not 
> use this parameter so that it remains the same.
> 
> Attached are the two most recent patches. I already updated the bugzilla 
> #55897 as well.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com] 
> Sent: Thursday, May 15, 2014 7:53 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
> We could either check for that during config/build or protect it with a 
> #ifdef in the code (and create some logging so the admin nows if it was found 
> or not).
> 
> On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:
> 
>> Hi Jim,
>> 
>> Thanks very much for your email.
>> 
>> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
>> file. If the feature is not supported on the OS (for example, Linux kernel < 
>> 3.9), it will fall back to the original behavior. 
>> 
>> In the bucket patch, there is no need to check the params. With single 
>> listen statement, it is just the default behavior. 
>> 
>> Please let me know if this answers your question.
>> 
>> Thanks,
>> Yingqi
>> 
>> -Original Message-----
>> From: Jim Jagielski [mailto:j...@jagunet.com]
>> Sent: Wednesday, May 14, 2014 6:57 AM
>> To: dev@httpd.apache.org
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>> SO_REUSEPORT support
>> 
>> This is very cool!
>> 
>> mod_status assumes that sysconf() exists, but do we need to do a config 
>> check on the params we use in these patches?
>> We look OK on Linux, FreeBSD and OSX...
>> 
>> I'm +1 on folding into trunk.
>> 
>> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
>> 
>>> Dear All,
>>> 
>>> During the last couple weeks, I spent some time extending the original two 
>>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
>>> event). Attached is the latest version of the two patches. Bugzilla 
>>> database has also been updated already. The ID for the two patches are 
>>> #55897 and #56279. Please refer to messages below for details on both of 
>>> the patches.
>>> 
>>> Quick test result on modern dual socket Intel platform (Linux Kernel
>>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
>>> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
>>> listen statements: 2.33X throughput improvement
>>> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 
>>> listen statements: 35% throughput improvement
>>> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 
>>> listen statements: throughput parity, but 62% response time reduction (with 
>>> patch, 38% response time as original SW)
>>> 
>>> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
>>> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
>>> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>>> 
>>> In all the above testing cases, significant response time reductions are 
>>> observed, even with throughput improvements.
>>> 
>>> Please let me know your feedback and comments.
>>> 
>>> Thanks,
>>> Yingqi
>>> Software and workloads used in performance tests may have been optimized 
>>> for performance only on Intel microprocessors. Performance tests, such as 
>>> SYSmark and MobileMark, are measured using specific computer systems, 
>>> components, software, operations and functions. Any change to any of those 
>>> factors may cause the results to vary. You should consult other information 
>>> and performance tests to assist you in fully evaluating your contemplated 
>>> purchases, including the performance of that product when combined with 
>>> other products.
>>> 
>>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>>> Sent: Monday, March 17, 2014 1:41 PM
>>> To: dev@httpd.apache.org
>>> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>>> SO_REUSEPORT support
>>> 
>>> Dear all,
>>> 
>>> 

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi All,

I just want to ping again on these two patches. 

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi [mailto:yingqi...@intel.com] 
Sent: Friday, May 23, 2014 9:03 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Dear All,

These two patches are the modified version based on the original single patch I 
sent out early January. Since I sent out the original patch, I have gotten lots 
of positive feedbacks and suggestions from this community. Thanks very much for 
your support and help!

The two most recent patches have addressed all the suggestions I got from you 
so far and they include changes for all three Linux MPMs. They have also been 
fully tested. Please refer to the change details and test results in the email 
below.

If there are no other comments/suggestions, should we fold them into the trunk?

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi [mailto:yingqi...@intel.com]
Sent: Friday, May 16, 2014 11:53 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Jim,

Thanks very much for clarifying this with me. I added #ifdef in the code to 
check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not use 
this parameter so that it remains the same.

Attached are the two most recent patches. I already updated the bugzilla #55897 
as well.

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Thursday, May 15, 2014 7:53 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
We could either check for that during config/build or protect it with a #ifdef 
in the code (and create some logging so the admin nows if it was found or not).

On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> Thanks very much for your email.
> 
> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
> file. If the feature is not supported on the OS (for example, Linux kernel < 
> 3.9), it will fall back to the original behavior. 
> 
> In the bucket patch, there is no need to check the params. With single listen 
> statement, it is just the default behavior. 
> 
> Please let me know if this answers your question.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com]
> Sent: Wednesday, May 14, 2014 6:57 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
> 
> This is very cool!
> 
> mod_status assumes that sysconf() exists, but do we need to do a config check 
> on the params we use in these patches?
> We look OK on Linux, FreeBSD and OSX...
> 
> I'm +1 on folding into trunk.
> 
> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
> 
>> Dear All,
>> 
>> During the last couple weeks, I spent some time extending the original two 
>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
>> event). Attached is the latest version of the two patches. Bugzilla database 
>> has also been updated already. The ID for the two patches are #55897 and 
>> #56279. Please refer to messages below for details on both of the patches.
>> 
>> Quick test result on modern dual socket Intel platform (Linux Kernel
>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
>> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
>> listen statements: 2.33X throughput improvement
>> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 
>> listen statements: 35% throughput improvement
>> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
>> statements: throughput parity, but 62% response time reduction (with patch, 
>> 38% response time as original SW)
>> 
>> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
>> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
>> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>> 
>> In all the above testing cases, significant response time reductions are 
>> observed, even with throughput improvements.
>> 
>> Please let me know your feedback and comments.
>> 
>> Thanks,
>> Yingqi
>> Software and workloads used in performance tests may have been optimized for 
>> performance only on Intel microprocessors. Performance tests, such as 
>> SYSmark and MobileMark, are measured using specific computer systems, 
>> components, software, operations and functions. Any change to any of those 
>> factors

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Dear All,

These two patches are the modified version based on the original single patch I 
sent out early January. Since I sent out the original patch, I have gotten lots 
of positive feedbacks and suggestions from this community. Thanks very much for 
your support and help!

The two most recent patches have addressed all the suggestions I got from you 
so far and they include changes for all three Linux MPMs. They have also been 
fully tested. Please refer to the change details and test results in the email 
below.

If there are no other comments/suggestions, should we fold them into the trunk?

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi [mailto:yingqi...@intel.com] 
Sent: Friday, May 16, 2014 11:53 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Jim,

Thanks very much for clarifying this with me. I added #ifdef in the code to 
check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not use 
this parameter so that it remains the same.

Attached are the two most recent patches. I already updated the bugzilla #55897 
as well.

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Thursday, May 15, 2014 7:53 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
We could either check for that during config/build or protect it with a #ifdef 
in the code (and create some logging so the admin nows if it was found or not).

On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> Thanks very much for your email.
> 
> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
> file. If the feature is not supported on the OS (for example, Linux kernel < 
> 3.9), it will fall back to the original behavior. 
> 
> In the bucket patch, there is no need to check the params. With single listen 
> statement, it is just the default behavior. 
> 
> Please let me know if this answers your question.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com]
> Sent: Wednesday, May 14, 2014 6:57 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
> 
> This is very cool!
> 
> mod_status assumes that sysconf() exists, but do we need to do a config check 
> on the params we use in these patches?
> We look OK on Linux, FreeBSD and OSX...
> 
> I'm +1 on folding into trunk.
> 
> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
> 
>> Dear All,
>> 
>> During the last couple weeks, I spent some time extending the original two 
>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
>> event). Attached is the latest version of the two patches. Bugzilla database 
>> has also been updated already. The ID for the two patches are #55897 and 
>> #56279. Please refer to messages below for details on both of the patches.
>> 
>> Quick test result on modern dual socket Intel platform (Linux Kernel
>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
>> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
>> listen statements: 2.33X throughput improvement
>> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 
>> listen statements: 35% throughput improvement
>> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
>> statements: throughput parity, but 62% response time reduction (with patch, 
>> 38% response time as original SW)
>> 
>> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
>> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
>> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>> 
>> In all the above testing cases, significant response time reductions are 
>> observed, even with throughput improvements.
>> 
>> Please let me know your feedback and comments.
>> 
>> Thanks,
>> Yingqi
>> Software and workloads used in performance tests may have been optimized for 
>> performance only on Intel microprocessors. Performance tests, such as 
>> SYSmark and MobileMark, are measured using specific computer systems, 
>> components, software, operations and functions. Any change to any of those 
>> factors may cause the results to vary. You should consult other information 
>> and performance tests to assist you in fully evaluating your contemplated 
>> purchases, including the performance of that product when combined with 
>> other products.
>> 
>> From: Lu, Yingqi 

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Dear All,

I am checking if there are any questions/comments on both of the patches? Also, 
I am wondering what the process of patch acceptance is.

Please let me know if there is anything I can do to help.

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi [mailto:yingqi...@intel.com] 
Sent: Friday, May 16, 2014 11:53 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Jim,

Thanks very much for clarifying this with me. I added #ifdef in the code to 
check _SC_NPROCESSORS_ONLN in the so_reuseport patch. Bucket patch does not use 
this parameter so that it remains the same.

Attached are the two most recent patches. I already updated the bugzilla #55897 
as well.

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Thursday, May 15, 2014 7:53 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
We could either check for that during config/build or protect it with a #ifdef 
in the code (and create some logging so the admin nows if it was found or not).

On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> Thanks very much for your email.
> 
> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
> file. If the feature is not supported on the OS (for example, Linux kernel < 
> 3.9), it will fall back to the original behavior. 
> 
> In the bucket patch, there is no need to check the params. With single listen 
> statement, it is just the default behavior. 
> 
> Please let me know if this answers your question.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com]
> Sent: Wednesday, May 14, 2014 6:57 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
> 
> This is very cool!
> 
> mod_status assumes that sysconf() exists, but do we need to do a config check 
> on the params we use in these patches?
> We look OK on Linux, FreeBSD and OSX...
> 
> I'm +1 on folding into trunk.
> 
> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
> 
>> Dear All,
>> 
>> During the last couple weeks, I spent some time extending the original two 
>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
>> event). Attached is the latest version of the two patches. Bugzilla database 
>> has also been updated already. The ID for the two patches are #55897 and 
>> #56279. Please refer to messages below for details on both of the patches.
>> 
>> Quick test result on modern dual socket Intel platform (Linux Kernel
>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
>> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
>> listen statements: 2.33X throughput improvement
>> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 
>> listen statements: 35% throughput improvement
>> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
>> statements: throughput parity, but 62% response time reduction (with patch, 
>> 38% response time as original SW)
>> 
>> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
>> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
>> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>> 
>> In all the above testing cases, significant response time reductions are 
>> observed, even with throughput improvements.
>> 
>> Please let me know your feedback and comments.
>> 
>> Thanks,
>> Yingqi
>> Software and workloads used in performance tests may have been optimized for 
>> performance only on Intel microprocessors. Performance tests, such as 
>> SYSmark and MobileMark, are measured using specific computer systems, 
>> components, software, operations and functions. Any change to any of those 
>> factors may cause the results to vary. You should consult other information 
>> and performance tests to assist you in fully evaluating your contemplated 
>> purchases, including the performance of that product when combined with 
>> other products.
>> 
>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>> Sent: Monday, March 17, 2014 1:41 PM
>> To: dev@httpd.apache.org
>> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>> SO_REUSEPORT support
>> 
>> Dear all,
>> 
>> Based on the feedback we received, we modified this patch. Here is the most 
>> recent version. We also modified the Bugzill

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Dear all,

Any other feedback/comments/questions? 

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi 
Sent: Wednesday, May 14, 2014 9:00 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Jim,

Thanks very much for your email.

In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
file. If the feature is not supported on the OS (for example, Linux kernel < 
3.9), it will fall back to the original behavior. 

In the bucket patch, there is no need to check the params. With single listen 
statement, it is just the default behavior. 

Please let me know if this answers your question.

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Wednesday, May 14, 2014 6:57 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

This is very cool!

mod_status assumes that sysconf() exists, but do we need to do a config check 
on the params we use in these patches?
We look OK on Linux, FreeBSD and OSX...

I'm +1 on folding into trunk.

On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:

> Dear All,
>  
> During the last couple weeks, I spent some time extending the original two 
> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
> event). Attached is the latest version of the two patches. Bugzilla database 
> has also been updated already. The ID for the two patches are #55897 and 
> #56279. Please refer to messages below for details on both of the patches.
>  
> Quick test result on modern dual socket Intel platform (Linux Kernel
> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
> listen statements: 2.33X throughput improvement
> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen 
> statements: 35% throughput improvement
> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
> statements: throughput parity, but 62% response time reduction (with patch, 
> 38% response time as original SW)
>  
> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>  
> In all the above testing cases, significant response time reductions are 
> observed, even with throughput improvements.
>  
> Please let me know your feedback and comments.
>  
> Thanks,
> Yingqi
> Software and workloads used in performance tests may have been optimized for 
> performance only on Intel microprocessors. Performance tests, such as SYSmark 
> and MobileMark, are measured using specific computer systems, components, 
> software, operations and functions. Any change to any of those factors may 
> cause the results to vary. You should consult other information and 
> performance tests to assist you in fully evaluating your contemplated 
> purchases, including the performance of that product when combined with other 
> products.
>  
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Monday, March 17, 2014 1:41 PM
> To: dev@httpd.apache.org
> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
>  
> Dear all,
>  
> Based on the feedback we received, we modified this patch. Here is the most 
> recent version. We also modified the Bugzilla database(Bugzilla# 55897 for 
> SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch).
>  
> Below are the changes we made into this new version:
>  
> According to Yann Ylavic and other people's comments, we separate the 
> original patch between with and without SO_REUSEPORT into two separated 
> patches. The SO_REUSEPORT patch does not change the original listen sockets, 
> it just duplicate the original one into multiple ones. Since the listen 
> sockets are identical, there is no need to change the idle_server_maintenance 
> function. The bucket patch (without SO_REUSEPORT), on the other hand, it 
> breaks down the original listen record (if there are multiple listen socks) 
> to multiple listen record linked lists. In this case, idle_server_maintenance 
> is implemented at bucket level to address the situation that imbalanced 
> traffic occurs among different listen sockets/children buckets. In the bucket 
> patch, the polling in the child process is removed since each child only 
> listens to 1 sock.
>  
> According to Arkadiusz Miskiewicz's comment, we make the "detection of 
> SO_REUSEPORT" at run time.
>  
> According to Jeff Trawick's comments,
> 1. We generate the patches against the httpd trunk.
> 2. We tested the current pa

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Dear all,

Any other feedback/comments/questions? 

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi 
Sent: Wednesday, May 14, 2014 9:00 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Jim,

Thanks very much for your email.

In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
file. If the feature is not supported on the OS (for example, Linux kernel < 
3.9), it will fall back to the original behavior. 

In the bucket patch, there is no need to check the params. With single listen 
statement, it is just the default behavior. 

Please let me know if this answers your question.

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Wednesday, May 14, 2014 6:57 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

This is very cool!

mod_status assumes that sysconf() exists, but do we need to do a config check 
on the params we use in these patches?
We look OK on Linux, FreeBSD and OSX...

I'm +1 on folding into trunk.

On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:

> Dear All,
>  
> During the last couple weeks, I spent some time extending the original two 
> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
> event). Attached is the latest version of the two patches. Bugzilla database 
> has also been updated already. The ID for the two patches are #55897 and 
> #56279. Please refer to messages below for details on both of the patches.
>  
> Quick test result on modern dual socket Intel platform (Linux Kernel
> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
> listen statements: 2.33X throughput improvement
> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen 
> statements: 35% throughput improvement
> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
> statements: throughput parity, but 62% response time reduction (with patch, 
> 38% response time as original SW)
>  
> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>  
> In all the above testing cases, significant response time reductions are 
> observed, even with throughput improvements.
>  
> Please let me know your feedback and comments.
>  
> Thanks,
> Yingqi
> Software and workloads used in performance tests may have been optimized for 
> performance only on Intel microprocessors. Performance tests, such as SYSmark 
> and MobileMark, are measured using specific computer systems, components, 
> software, operations and functions. Any change to any of those factors may 
> cause the results to vary. You should consult other information and 
> performance tests to assist you in fully evaluating your contemplated 
> purchases, including the performance of that product when combined with other 
> products.
>  
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Monday, March 17, 2014 1:41 PM
> To: dev@httpd.apache.org
> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
>  
> Dear all,
>  
> Based on the feedback we received, we modified this patch. Here is the most 
> recent version. We also modified the Bugzilla database(Bugzilla# 55897 for 
> SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch).
>  
> Below are the changes we made into this new version:
>  
> According to Yann Ylavic and other people's comments, we separate the 
> original patch between with and without SO_REUSEPORT into two separated 
> patches. The SO_REUSEPORT patch does not change the original listen sockets, 
> it just duplicate the original one into multiple ones. Since the listen 
> sockets are identical, there is no need to change the idle_server_maintenance 
> function. The bucket patch (without SO_REUSEPORT), on the other hand, it 
> breaks down the original listen record (if there are multiple listen socks) 
> to multiple listen record linked lists. In this case, idle_server_maintenance 
> is implemented at bucket level to address the situation that imbalanced 
> traffic occurs among different listen sockets/children buckets. In the bucket 
> patch, the polling in the child process is removed since each child only 
> listens to 1 sock.
>  
> According to Arkadiusz Miskiewicz's comment, we make the "detection of 
> SO_REUSEPORT" at run time.
>  
> According to Jeff Trawick's comments,
> 1. We generate the patches against the httpd trunk.
> 2. We tested the current pa

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Dear all,

I sent this message twice in the morning, but I did not receive anything back. 
I am resending this one more time, just trying to gather your 
feedback/comments/questions on both of the patches. 

Sorry for the duplications.

Thanks,
Yingqi

-Original Message-
From: Lu, Yingqi
Sent: Wednesday, May 14, 2014 9:00 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Jim,

Thanks very much for your email.

In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
file. If the feature is not supported on the OS (for example, Linux kernel < 
3.9), it will fall back to the original behavior. 

In the bucket patch, there is no need to check the params. With single listen 
statement, it is just the default behavior. 

Please let me know if this answers your question.

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Wednesday, May 14, 2014 6:57 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

This is very cool!

mod_status assumes that sysconf() exists, but do we need to do a config check 
on the params we use in these patches?
We look OK on Linux, FreeBSD and OSX...

I'm +1 on folding into trunk.

On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:

> Dear All,
>  
> During the last couple weeks, I spent some time extending the original two 
> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
> event). Attached is the latest version of the two patches. Bugzilla database 
> has also been updated already. The ID for the two patches are #55897 and 
> #56279. Please refer to messages below for details on both of the patches.
>  
> Quick test result on modern dual socket Intel platform (Linux Kernel
> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
> listen statements: 2.33X throughput improvement
> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen 
> statements: 35% throughput improvement
> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
> statements: throughput parity, but 62% response time reduction (with patch, 
> 38% response time as original SW)
>  
> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>  
> In all the above testing cases, significant response time reductions are 
> observed, even with throughput improvements.
>  
> Please let me know your feedback and comments.
>  
> Thanks,
> Yingqi
> Software and workloads used in performance tests may have been optimized for 
> performance only on Intel microprocessors. Performance tests, such as SYSmark 
> and MobileMark, are measured using specific computer systems, components, 
> software, operations and functions. Any change to any of those factors may 
> cause the results to vary. You should consult other information and 
> performance tests to assist you in fully evaluating your contemplated 
> purchases, including the performance of that product when combined with other 
> products.
>  
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Monday, March 17, 2014 1:41 PM
> To: dev@httpd.apache.org
> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
>  
> Dear all,
>  
> Based on the feedback we received, we modified this patch. Here is the most 
> recent version. We also modified the Bugzilla database(Bugzilla# 55897 for 
> SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch).
>  
> Below are the changes we made into this new version:
>  
> According to Yann Ylavic and other people's comments, we separate the 
> original patch between with and without SO_REUSEPORT into two separated 
> patches. The SO_REUSEPORT patch does not change the original listen sockets, 
> it just duplicate the original one into multiple ones. Since the listen 
> sockets are identical, there is no need to change the idle_server_maintenance 
> function. The bucket patch (without SO_REUSEPORT), on the other hand, it 
> breaks down the original listen record (if there are multiple listen socks) 
> to multiple listen record linked lists. In this case, idle_server_maintenance 
> is implemented at bucket level to address the situation that imbalanced 
> traffic occurs among different listen sockets/children buckets. In the bucket 
> patch, the polling in the child process is removed since each child only 
> listens to 1 sock.
>  
> According to Arkadiusz Miskiewicz's comment, we make the "detection of 

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

I was thinking more about the sysconf(_SC_NPROCESSORS_ONLN) stuff...
We could either check for that during config/build or protect
it with a #ifdef in the code (and create some logging so
the admin nows if it was found or not).

On May 14, 2014, at 11:59 AM, Lu, Yingqi  wrote:

> Hi Jim,
> 
> Thanks very much for your email.
> 
> In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
> file. If the feature is not supported on the OS (for example, Linux kernel < 
> 3.9), it will fall back to the original behavior. 
> 
> In the bucket patch, there is no need to check the params. With single listen 
> statement, it is just the default behavior. 
> 
> Please let me know if this answers your question.
> 
> Thanks,
> Yingqi
> 
> -Original Message-
> From: Jim Jagielski [mailto:j...@jagunet.com] 
> Sent: Wednesday, May 14, 2014 6:57 AM
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
> support
> 
> This is very cool!
> 
> mod_status assumes that sysconf() exists, but do we need to do a config check 
> on the params we use in these patches?
> We look OK on Linux, FreeBSD and OSX...
> 
> I'm +1 on folding into trunk.
> 
> On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:
> 
>> Dear All,
>> 
>> During the last couple weeks, I spent some time extending the original two 
>> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
>> event). Attached is the latest version of the two patches. Bugzilla database 
>> has also been updated already. The ID for the two patches are #55897 and 
>> #56279. Please refer to messages below for details on both of the patches.
>> 
>> Quick test result on modern dual socket Intel platform (Linux Kernel 
>> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
>> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
>> listen statements: 2.33X throughput improvement
>> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 
>> listen statements: 35% throughput improvement
>> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
>> statements: throughput parity, but 62% response time reduction (with patch, 
>> 38% response time as original SW)
>> 
>> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
>> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
>> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>> 
>> In all the above testing cases, significant response time reductions are 
>> observed, even with throughput improvements.
>> 
>> Please let me know your feedback and comments.
>> 
>> Thanks,
>> Yingqi
>> Software and workloads used in performance tests may have been optimized for 
>> performance only on Intel microprocessors. Performance tests, such as 
>> SYSmark and MobileMark, are measured using specific computer systems, 
>> components, software, operations and functions. Any change to any of those 
>> factors may cause the results to vary. You should consult other information 
>> and performance tests to assist you in fully evaluating your contemplated 
>> purchases, including the performance of that product when combined with 
>> other products.
>> 
>> From: Lu, Yingqi [mailto:yingqi...@intel.com]
>> Sent: Monday, March 17, 2014 1:41 PM
>> To: dev@httpd.apache.org
>> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
>> SO_REUSEPORT support
>> 
>> Dear all,
>> 
>> Based on the feedback we received, we modified this patch. Here is the most 
>> recent version. We also modified the Bugzilla database(Bugzilla# 55897 for 
>> SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch).
>> 
>> Below are the changes we made into this new version:
>> 
>> According to Yann Ylavic and other people's comments, we separate the 
>> original patch between with and without SO_REUSEPORT into two separated 
>> patches. The SO_REUSEPORT patch does not change the original listen sockets, 
>> it just duplicate the original one into multiple ones. Since the listen 
>> sockets are identical, there is no need to change the 
>> idle_server_maintenance function. The bucket patch (without SO_REUSEPORT), 
>> on the other hand, it breaks down the original listen record (if there are 
>> multiple listen socks) to multiple listen record linked lists. In this case, 
>> idle_server_maintenance is implemented at bucket level to address the 
>> situation that imbalanced traffic occurs among different liste

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Now, I just got the message I sent out yesterday, what a long delay!

Actually, this morning I sent out several messages checking for other 
feedback/comments/questions. I guess those messages are still in the air since 
I have not received any copy of those mails.

All, please let me know your feedback/comments/questions. Thanks very much for 
your time reviewing the patches.

Thanks,
Yingqi 

-Original Message-
From: Lu, Yingqi [mailto:yingqi...@intel.com] 
Sent: Wednesday, May 14, 2014 9:00 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Jim,

Thanks very much for your email.

In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
file. If the feature is not supported on the OS (for example, Linux kernel < 
3.9), it will fall back to the original behavior. 

In the bucket patch, there is no need to check the params. With single listen 
statement, it is just the default behavior. 

Please let me know if this answers your question.

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com]
Sent: Wednesday, May 14, 2014 6:57 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

This is very cool!

mod_status assumes that sysconf() exists, but do we need to do a config check 
on the params we use in these patches?
We look OK on Linux, FreeBSD and OSX...

I'm +1 on folding into trunk.

On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:

> Dear All,
>  
> During the last couple weeks, I spent some time extending the original two 
> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
> event). Attached is the latest version of the two patches. Bugzilla database 
> has also been updated already. The ID for the two patches are #55897 and 
> #56279. Please refer to messages below for details on both of the patches.
>  
> Quick test result on modern dual socket Intel platform (Linux Kernel
> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
> listen statements: 2.33X throughput improvement
> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen 
> statements: 35% throughput improvement
> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
> statements: throughput parity, but 62% response time reduction (with patch, 
> 38% response time as original SW)
>  
> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>  
> In all the above testing cases, significant response time reductions are 
> observed, even with throughput improvements.
>  
> Please let me know your feedback and comments.
>  
> Thanks,
> Yingqi
> Software and workloads used in performance tests may have been optimized for 
> performance only on Intel microprocessors. Performance tests, such as SYSmark 
> and MobileMark, are measured using specific computer systems, components, 
> software, operations and functions. Any change to any of those factors may 
> cause the results to vary. You should consult other information and 
> performance tests to assist you in fully evaluating your contemplated 
> purchases, including the performance of that product when combined with other 
> products.
>  
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Monday, March 17, 2014 1:41 PM
> To: dev@httpd.apache.org
> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
>  
> Dear all,
>  
> Based on the feedback we received, we modified this patch. Here is the most 
> recent version. We also modified the Bugzilla database(Bugzilla# 55897 for 
> SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch).
>  
> Below are the changes we made into this new version:
>  
> According to Yann Ylavic and other people's comments, we separate the 
> original patch between with and without SO_REUSEPORT into two separated 
> patches. The SO_REUSEPORT patch does not change the original listen sockets, 
> it just duplicate the original one into multiple ones. Since the listen 
> sockets are identical, there is no need to change the idle_server_maintenance 
> function. The bucket patch (without SO_REUSEPORT), on the other hand, it 
> breaks down the original listen record (if there are multiple listen socks) 
> to multiple listen record linked lists. In this case, idle_server_maintenance 
> is implemented at bucket level to address the situation that imbalanced 
> traffic occurs among different listen sockets/children buckets. In the bucket 
> patch, the poll

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Jim,

Thanks very much for your email.

In the SO_REUSEPORT patch, SO_REUSEPORT support is checked inside listen.c 
file. If the feature is not supported on the OS (for example, Linux kernel < 
3.9), it will fall back to the original behavior. 

In the bucket patch, there is no need to check the params. With single listen 
statement, it is just the default behavior. 

Please let me know if this answers your question.

Thanks,
Yingqi

-Original Message-
From: Jim Jagielski [mailto:j...@jagunet.com] 
Sent: Wednesday, May 14, 2014 6:57 AM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

This is very cool!

mod_status assumes that sysconf() exists, but do we need to do a config check 
on the params we use in these patches?
We look OK on Linux, FreeBSD and OSX...

I'm +1 on folding into trunk.

On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:

> Dear All,
>  
> During the last couple weeks, I spent some time extending the original two 
> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
> event). Attached is the latest version of the two patches. Bugzilla database 
> has also been updated already. The ID for the two patches are #55897 and 
> #56279. Please refer to messages below for details on both of the patches.
>  
> Quick test result on modern dual socket Intel platform (Linux Kernel 
> 3.13.9) SO_REUSEPORT patch (bugzilla #55897)
> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
> listen statements: 2.33X throughput improvement
> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen 
> statements: 35% throughput improvement
> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
> statements: throughput parity, but 62% response time reduction (with patch, 
> 38% response time as original SW)
>  
> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>  
> In all the above testing cases, significant response time reductions are 
> observed, even with throughput improvements.
>  
> Please let me know your feedback and comments.
>  
> Thanks,
> Yingqi
> Software and workloads used in performance tests may have been optimized for 
> performance only on Intel microprocessors. Performance tests, such as SYSmark 
> and MobileMark, are measured using specific computer systems, components, 
> software, operations and functions. Any change to any of those factors may 
> cause the results to vary. You should consult other information and 
> performance tests to assist you in fully evaluating your contemplated 
> purchases, including the performance of that product when combined with other 
> products.
>  
> From: Lu, Yingqi [mailto:yingqi...@intel.com]
> Sent: Monday, March 17, 2014 1:41 PM
> To: dev@httpd.apache.org
> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
>  
> Dear all,
>  
> Based on the feedback we received, we modified this patch. Here is the most 
> recent version. We also modified the Bugzilla database(Bugzilla# 55897 for 
> SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch).
>  
> Below are the changes we made into this new version:
>  
> According to Yann Ylavic and other people's comments, we separate the 
> original patch between with and without SO_REUSEPORT into two separated 
> patches. The SO_REUSEPORT patch does not change the original listen sockets, 
> it just duplicate the original one into multiple ones. Since the listen 
> sockets are identical, there is no need to change the idle_server_maintenance 
> function. The bucket patch (without SO_REUSEPORT), on the other hand, it 
> breaks down the original listen record (if there are multiple listen socks) 
> to multiple listen record linked lists. In this case, idle_server_maintenance 
> is implemented at bucket level to address the situation that imbalanced 
> traffic occurs among different listen sockets/children buckets. In the bucket 
> patch, the polling in the child process is removed since each child only 
> listens to 1 sock.
>  
> According to Arkadiusz Miskiewicz's comment, we make the "detection of 
> SO_REUSEPORT" at run time.
>  
> According to Jeff Trawick's comments,
> 1. We generate the patches against the httpd trunk.
> 2. We tested the current patches and they do not impact event and worker 
> mpms. If current patches can be accepted, we would be happy to extend them to 
> other Linux based mpms. There are not much code changes, but require some 
> time to setup the workload to test.
> 3.

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

This is very cool!

mod_status assumes that sysconf() exists, but do we need
to do a config check on the params we use in these patches?
We look OK on Linux, FreeBSD and OSX...

I'm +1 on folding into trunk.

On May 13, 2014, at 7:55 PM, Lu, Yingqi  wrote:

> Dear All,
>  
> During the last couple weeks, I spent some time extending the original two 
> patches from prefork MPM only to all three Linux MPMs (prefork, worker and 
> event). Attached is the latest version of the two patches. Bugzilla database 
> has also been updated already. The ID for the two patches are #55897 and 
> #56279. Please refer to messages below for details on both of the patches.
>  
> Quick test result on modern dual socket Intel platform (Linux Kernel 3.13.9)
> SO_REUSEPORT patch (bugzilla #55897)
> 1.   Prefork MPM: 1 listen statement: 2.16X throughput improvement; 2 
> listen statements: 2.33X throughput improvement
> 2.   Worker MPM: 1 listen statement: 10% throughput improvement; 2 listen 
> statements: 35% throughput improvement
> 3.   Event MPM: 1 listen statement: 13% throughput improvement; 2 listen 
> statements: throughput parity, but 62% response time reduction (with patch, 
> 38% response time as original SW)
>  
> Bucket patch (bugzilla #56279, only impact multiple listen statement case)
> 1.   Prefork MPM: 2 listen statements: 42% throughput improvement
> 2.   Worker MPM: 2 listen statements: 7% throughput improvement
>  
> In all the above testing cases, significant response time reductions are 
> observed, even with throughput improvements.
>  
> Please let me know your feedback and comments.
>  
> Thanks,
> Yingqi
> Software and workloads used in performance tests may have been optimized for 
> performance only on Intel microprocessors. Performance tests, such as SYSmark 
> and MobileMark, are measured using specific computer systems, components, 
> software, operations and functions. Any change to any of those factors may 
> cause the results to vary. You should consult other information and 
> performance tests to assist you in fully evaluating your contemplated 
> purchases, including the performance of that product when combined with other 
> products.
>  
> From: Lu, Yingqi [mailto:yingqi...@intel.com] 
> Sent: Monday, March 17, 2014 1:41 PM
> To: dev@httpd.apache.org
> Subject: RE: FW: [PATCH ASF bugzilla# 55897]prefork_mpm patch with 
> SO_REUSEPORT support
>  
> Dear all,
>  
> Based on the feedback we received, we modified this patch. Here is the most 
> recent version. We also modified the Bugzilla database(Bugzilla# 55897 for 
> SO_REUSEPORT patch; Bugzilla# 56279 for bucket patch).
>  
> Below are the changes we made into this new version:
>  
> According to Yann Ylavic and other people’s comments, we separate the 
> original patch between with and without SO_REUSEPORT into two separated 
> patches. The SO_REUSEPORT patch does not change the original listen sockets, 
> it just duplicate the original one into multiple ones. Since the listen 
> sockets are identical, there is no need to change the idle_server_maintenance 
> function. The bucket patch (without SO_REUSEPORT), on the other hand, it 
> breaks down the original listen record (if there are multiple listen socks) 
> to multiple listen record linked lists. In this case, idle_server_maintenance 
> is implemented at bucket level to address the situation that imbalanced 
> traffic occurs among different listen sockets/children buckets. In the bucket 
> patch, the polling in the child process is removed since each child only 
> listens to 1 sock.
>  
> According to Arkadiusz Miskiewicz’s comment, we make the “detection of 
> SO_REUSEPORT” at run time.
>  
> According to Jeff Trawick’s comments,
> 1. We generate the patches against the httpd trunk.
> 2. We tested the current patches and they do not impact event and worker 
> mpms. If current patches can be accepted, we would be happy to extend them to 
> other Linux based mpms. There are not much code changes, but require some 
> time to setup the workload to test.
> 3. We removed unnecessary comments and changed APLOGNO(). We also changed 
> some of the parameter/variable/function names to better represent their 
> meanings.
> 4. There should be no build-in limitations for SO_REUSEPORT patch. For bucket 
> patch, the only thing is the number of children bucket only scales to 
> MAX_SPAWN_RATE. If there are more than 32 (current default MAX_SPQN_RATE) 
> listen statements specified in the httpd.conf, the number of buckets will be 
> fixed to 32. The reason for this is because that we implement the 
> idle_server_maintenance at bucket level, each bucket’s own max_spawn_rate is 
> set to MAX_SPAWN_RATE/num_buckets.
>  
> Again, thanks very much for all the comments and feedback. Please let us know 
> if there are more changes we need to complete to make them accepted.
>  
> Thanks,
> Yingqi Lu
>  
>  
>  
> From: Lu, Yingqi 
> Sent: Tuesday, March 04, 2014 10:43 AM
> To: dev@httpd.

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Thanks, Graham! I am looking forward to hearing your feedback.

Thanks,
Yingqi

From: Graham Leggett [mailto:minf...@sharp.fm]
Sent: Monday, April 07, 2014 12:08 PM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

On 07 Apr 2014, at 6:21 PM, "Lu, Yingqi" 
mailto:yingqi...@intel.com>> wrote:

I just want to ping again on the modifications we made on both of the patches 
[bugzilla #55897 and bugzilla #56279]. Please let us know your comments and 
feedback.

I am reattaching the patch files here in case you missed original email.

I am very keen to review this, but have no time right now - sorry about that. 
From my side I am keen to review it soon.

Regards,
Graham
--

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Graham Leggett]

On 07 Apr 2014, at 6:21 PM, "Lu, Yingqi"  wrote:

> I just want to ping again on the modifications we made on both of the patches 
> [bugzilla #55897 and bugzilla #56279]. Please let us know your comments and 
> feedback.
>  
> I am reattaching the patch files here in case you missed original email.

I am very keen to review this, but have no time right now - sorry about that. 
From my side I am keen to review it soon.

Regards,
Graham
--

RE: [PATCH ASF bugzilla# 55897] prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Tim,

Thanks for your email. 

SO_REUSEPORT feature is enabled on Linux kernel 3.9 and newer. The feature is 
defined at /usr/include/asm-generic/socket.h. 

With the old kernel, the definition is there, but is commented out. 
/*#define SO_REUSEPORT  15*/

The section of code below is just to define SO_REUSEPORT if it is not already 
being defined. The code after this is to detect if SO_REUSEPORT is supported or 
not.

I am using x86_64 systems with Linux. If anyone finds something different on 
your system, please let me know.

Thanks,
Yingqi

-Original Message-
From: Tim Bannister [mailto:is...@jellybaby.net] 
Sent: Monday, March 17, 2014 2:31 PM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897] prefork_mpm patch with SO_REUSEPORT 
support

I'm afraid I don't understand this particular part from 
httpd_trunk_so_reuseport.patch:

#ifndef SO_REUSEPORT
#define SO_REUSEPORT 15
#endif

Why 15? Is this going to be portable across different platforms?

-- 
Tim Bannister - is...@jellybaby.net

Re: [PATCH ASF bugzilla# 55897] prefork_mpm patch with SO_REUSEPORT support

[Tim Bannister]

I'm afraid I don't understand this particular part from 
httpd_trunk_so_reuseport.patch:

#ifndef SO_REUSEPORT
#define SO_REUSEPORT 15
#endif

Why 15? Is this going to be portable across different platforms?

-- 
Tim Bannister – is...@jellybaby.net

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Yann,

Yes, without SO_REUSEPORT, child only accepts connections from a single 
listening socket only. In order to address the situation of in-balanced traffic 
among different sockets/listen statements, the patch makes each bucket does its 
own idler server maintenance. For example, if we have two listen statements 
defined, one is very busy and the other is almost idle. The patch creates two 
buckets, each listens to 1 IP:port. The busy bucket would end up with lots of 
children and idle bucket would only maintain minimum number of children which 
is equal to 1/2 of the min idle servers defined in the httpd.conf file.

Thanks,
Yingqi

From: Yann Ylavic [mailto:ylavic@gmail.com]
Sent: Thursday, March 06, 2014 5:49 AM
To: httpd
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

On Wed, Mar 5, 2014 at 6:38 PM, Lu, Yingqi 
mailto:yingqi...@intel.com>> wrote:

1. If I understand correctly (please correct me if not), do you suggest 
duplicating the listen socks inside the child process with SO_REUSEPROT 
enabled? Yes, I agree this would be a cleaner implementation and I actually 
tried that before. However, I encountered the "connection reset" error since 
the number of the child process is changing. I googled online and found it 
actually being discussed here at http://lwn.net/Articles/542629/.

Actually I found that article, but expected the "defect" was solved since 
then...
This looks like a thorn in the side of MPMs in general,
but couldn't find any pointer to a fix, do you know if there is some progress 
on this in the latest linux kernel?

For testing purpose (until then?), you could also configure MPM prefork to not 
create/terminate children processes once started (using the same value for 
StartServers and ServerLimit, still MaxRequetsPerChild 0).
It could be interesting to see how SO_REUSEPORT scales in these optimal 
conditions (no lock, full OS round-robin on all listeners).
For this you would have to use your former patch (duplicating listeners in each 
child process), and do nothing in SAFE_ACCEPT when HAVE_SO_REUSEPORT.
Also, SO_REUSEPORT exists on (and even comes from) FreeBSD if I am not 
mistaken, but it seems that there is no round-robin garantee for it, is there? 
Could this patch also take advantage of BSD's SO_REUSEPORT implementation?


2. Then, I decided to do the socket duplication in the parent process. The goal 
of this change is to extend the CPU thread scalability with the big thread 
count system. Therefore, I just very simply defined 
number_of_listen_buckets=total_number_active_thread/8, and each listen bucket 
has a dedicated listener. I do not want to over duplicate the socket; 
otherwise, it would create too many child processes at the beginning. One 
listen bucket should have at least one child process to start with. However, 
this is only my understanding and it may not be correct and complete. If you 
have other ideas, please share with us. Feedbacks and comments are very welcome 
here :)

The listeners buckets make sense with SO_REUSEPORT given the defect, I hope 
this is temporary.


3. I am struggling with myself as well on if we should put with and without 
SO_REUSEPORT into two different patches. The only reason I put them together is 
because they both use the concept of listen buckets. If you think it would make 
more sense to separate them into two patches, I can certainly do that. Also, I 
am a little bit confused about your comments "On the other hand, each child is 
dedicated, won't one have to multiply the configured ServerLimit by the number 
of Listen to achieve the same (maximum theoretical) scalability with regard to 
all the listeners?". Can you please explain a little bit more on this? Really 
appreciate.

Sorry to have not been clear enough (nay at all).

I'm referring to the following code.

In prefork.c::make_child(), each child is assigned a listener like this (before 
fork()ing) :

child_listen = mpm_listen[bucket[slot]];

and then each child will use child_listen as listeners list.

The duplicated listeners array (mpm_listen) is built by the following (new) 
function :

/* This function is added for the patch. This function duplicates
 * open_listeners, alloc_listener() and re-call make_sock() for the
 * duplicated listeners. In this function, the newly created sockets
 * will bind and listen*/
AP_DECLARE(apr_status_t) ap_post_config_listeners(server_rec *s, apr_pool_t *p,
  int num_buckets) {
mpm_listen = apr_palloc(p, sizeof(ap_listen_rec*) * num_buckets);
int i;
ap_listen_rec *lr;
/* duplicate from alloc_listener() for the additional listen record*/
lr = ap_listeners;
for (i = 0; i < num_buckets; i++) {
#ifdef HAVE_SO_REUSEPORT
ap_listen_rec *templr;
ap_listen_rec *last = NULL;
while (lr) {
templr = ap_duplicate_listener(p, lr);

  

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Yann Ylavic]

Hi Bill,

I was just worried about forking mpm_prefork into mpm_prefork_buckets, and
so on with worker/event/..., most of the code would have been the same.

But I can't disagree with you, factorizing the existing MPMs shared codes
(it seems there are quite some) and future ones into a common interface is
indeed the way to go.

+1 for as much MPM as needed provided minimal code is duplicated.

Best regards,
Yann.


On Thu, Mar 6, 2014 at 5:49 PM, William A. Rowe Jr. wrote:

> Yann,
>
> what you might wish to consider is that each individual MPM may be
> compiled alongside the others.  If you do a feature select, you are
> left with one of the other.
>
> If it is designed to cohabitate, then it may share sources under the
> os/ branch, but can still exist as a separate loadable MPM.  Any new
> MPM idea with a proper ICLA/CCLA/Code Grant, such that our users can
> make the decision for themselves, I expect will be swiftly tested,
> benchmarked and perhaps adopted.
>
> If it is to exist as a replacement of an MPM in the stable 2.4 tree, I
> expect it will meet with stiff resistance.  Let's work out an
> appropriate way to adopt the code?
>
> Respectfully,
>
> Bill
>
> On Thu, Mar 6, 2014 at 8:26 AM, Yann Ylavic  wrote:
> > I'm not sure (yet) a new MPM is needed, or rather, multiple new MPMs are
> > needed.
> >
> > The "bucketized" listeners is applyable to all (*nix only?) MPMs, that
> would
> > lead to as much forks...
> > Couldn't new directives be created instead (ServerBucketsNum, Listen
> >  , ...), defaulting to the current behaviour?
> >
> >
> > On Thu, Mar 6, 2014 at 2:49 PM, Jim Jagielski  wrote:
> >>
> >> ++1.
> >>
> >>
> >> On Mar 6, 2014, at 3:15 AM, Plüm, Rüdiger, Vodafone Group
> >>  wrote:
> >>
> >> >
> >> >
> >> >> -Original Message-
> >> >> From: William A. Rowe Jr. [mailto:wmr...@gmail.com]
> >> >> Sent: Donnerstag, 6. März 2014 06:58
> >> >> To: dev@httpd.apache.org
> >> >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
> >> >> SO_REUSEPORT support
> >> >>
> >> >>
> >> >> If you want to truly re-architect the MPM, by all means, propose it
> as
> >> >> another MPM module.  If it isn't adopted here, please don't hesitate
> >> >> to offer it to interested users as separate source (although I hope
> we
> >> >> find a way to adopt it.)
> >> >>
> >> >> The idea of different MPM's was that they were swappable.  MPM foo
> >> >> isn't MPM bar.  E.g., worker, prefork, event each have their own
> tree.
> >> >> Likewise, there is nothing stopping us from having 2, or 3 MPM's on
> >> >> Windows, and there is nothing stopping us from stating that there is
> a
> >> >> prerequisite on a particular MPM of Linux 3.1 kernels or Windows
> >> >> 2008+.
> >> >
> >> > +1 to a new MPM on trunk. This gives it more time to settle and to
> >> > stabilize
> >> > without disrupting current stuff. And if it is fast and stable it will
> >> > certainly
> >> > cause the 'older' MPM to drop in userbase :-).
> >> > IMHO this would even open a path to 2.4.x provided that we do not need
> >> > any other
> >> > non backportable changes to do this.
> >> >
> >> > Regards
> >> >
> >> > Rüdiger
> >> >
> >>
> >
>

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[William A. Rowe Jr.]

Yann,

what you might wish to consider is that each individual MPM may be
compiled alongside the others.  If you do a feature select, you are
left with one of the other.

If it is designed to cohabitate, then it may share sources under the
os/ branch, but can still exist as a separate loadable MPM.  Any new
MPM idea with a proper ICLA/CCLA/Code Grant, such that our users can
make the decision for themselves, I expect will be swiftly tested,
benchmarked and perhaps adopted.

If it is to exist as a replacement of an MPM in the stable 2.4 tree, I
expect it will meet with stiff resistance.  Let's work out an
appropriate way to adopt the code?

Respectfully,

Bill

On Thu, Mar 6, 2014 at 8:26 AM, Yann Ylavic  wrote:
> I'm not sure (yet) a new MPM is needed, or rather, multiple new MPMs are
> needed.
>
> The "bucketized" listeners is applyable to all (*nix only?) MPMs, that would
> lead to as much forks...
> Couldn't new directives be created instead (ServerBucketsNum, Listen
>  , ...), defaulting to the current behaviour?
>
>
> On Thu, Mar 6, 2014 at 2:49 PM, Jim Jagielski  wrote:
>>
>> ++1.
>>
>>
>> On Mar 6, 2014, at 3:15 AM, Plüm, Rüdiger, Vodafone Group
>>  wrote:
>>
>> >
>> >
>> >> -Original Message-
>> >> From: William A. Rowe Jr. [mailto:wmr...@gmail.com]
>> >> Sent: Donnerstag, 6. März 2014 06:58
>> >> To: dev@httpd.apache.org
>> >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>> >> SO_REUSEPORT support
>> >>
>> >>
>> >> If you want to truly re-architect the MPM, by all means, propose it as
>> >> another MPM module.  If it isn't adopted here, please don't hesitate
>> >> to offer it to interested users as separate source (although I hope we
>> >> find a way to adopt it.)
>> >>
>> >> The idea of different MPM's was that they were swappable.  MPM foo
>> >> isn't MPM bar.  E.g., worker, prefork, event each have their own tree.
>> >> Likewise, there is nothing stopping us from having 2, or 3 MPM's on
>> >> Windows, and there is nothing stopping us from stating that there is a
>> >> prerequisite on a particular MPM of Linux 3.1 kernels or Windows
>> >> 2008+.
>> >
>> > +1 to a new MPM on trunk. This gives it more time to settle and to
>> > stabilize
>> > without disrupting current stuff. And if it is fast and stable it will
>> > certainly
>> > cause the 'older' MPM to drop in userbase :-).
>> > IMHO this would even open a path to 2.4.x provided that we do not need
>> > any other
>> > non backportable changes to do this.
>> >
>> > Regards
>> >
>> > Rüdiger
>> >
>>
>

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Yann Ylavic]

I'm not sure (yet) a new MPM is needed, or rather, multiple new MPMs are
needed.

The "bucketized" listeners is applyable to all (*nix only?) MPMs, that
would lead to as much forks...
Couldn't new directives be created instead (ServerBucketsNum, Listen
 , ...), defaulting to the current behaviour?


On Thu, Mar 6, 2014 at 2:49 PM, Jim Jagielski  wrote:

> ++1.
>
>
> On Mar 6, 2014, at 3:15 AM, Plüm, Rüdiger, Vodafone Group <
> ruediger.pl...@vodafone.com> wrote:
>
> >
> >
> >> -Original Message-
> >> From: William A. Rowe Jr. [mailto:wmr...@gmail.com]
> >> Sent: Donnerstag, 6. März 2014 06:58
> >> To: dev@httpd.apache.org
> >> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
> >> SO_REUSEPORT support
> >>
> >>
> >> If you want to truly re-architect the MPM, by all means, propose it as
> >> another MPM module.  If it isn't adopted here, please don't hesitate
> >> to offer it to interested users as separate source (although I hope we
> >> find a way to adopt it.)
> >>
> >> The idea of different MPM's was that they were swappable.  MPM foo
> >> isn't MPM bar.  E.g., worker, prefork, event each have their own tree.
> >> Likewise, there is nothing stopping us from having 2, or 3 MPM's on
> >> Windows, and there is nothing stopping us from stating that there is a
> >> prerequisite on a particular MPM of Linux 3.1 kernels or Windows
> >> 2008+.
> >
> > +1 to a new MPM on trunk. This gives it more time to settle and to
> stabilize
> > without disrupting current stuff. And if it is fast and stable it will
> certainly
> > cause the 'older' MPM to drop in userbase :-).
> > IMHO this would even open a path to 2.4.x provided that we do not need
> any other
> > non backportable changes to do this.
> >
> > Regards
> >
> > Rüdiger
> >
>
>

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[William A. Rowe Jr.]

On Wed, Mar 5, 2014 at 11:38 AM, Lu, Yingqi  wrote:
> 1. If I understand correctly (please correct me if not), do you suggest
> duplicating the listen socks inside the child process with SO_REUSEPROT
> enabled? Yes, I agree this would be a cleaner implementation and I actually
> tried that before. However, I encountered the "connection reset" error since
> the number of the child process is changing. I googled online and found it
> actually being discussed here at http://lwn.net/Articles/542629/.

You actually need to refer to Unix Network Programming, Vol 1 by W
Richard Stevens to have a context to ask the question.

Apologies if that comes across as harsh, but either Stevens is wrong,
or the kernel developers are wrong.  It is unlikely that both are
wrong and very likely that both are not right.  Sometimes we just have
to send kernel bugs upstream.

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Jim Jagielski]

++1.


On Mar 6, 2014, at 3:15 AM, Plüm, Rüdiger, Vodafone Group 
 wrote:

> 
> 
>> -Original Message-
>> From: William A. Rowe Jr. [mailto:wmr...@gmail.com]
>> Sent: Donnerstag, 6. März 2014 06:58
>> To: dev@httpd.apache.org
>> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
>> SO_REUSEPORT support
>> 
>> 
>> If you want to truly re-architect the MPM, by all means, propose it as
>> another MPM module.  If it isn't adopted here, please don't hesitate
>> to offer it to interested users as separate source (although I hope we
>> find a way to adopt it.)
>> 
>> The idea of different MPM's was that they were swappable.  MPM foo
>> isn't MPM bar.  E.g., worker, prefork, event each have their own tree.
>> Likewise, there is nothing stopping us from having 2, or 3 MPM's on
>> Windows, and there is nothing stopping us from stating that there is a
>> prerequisite on a particular MPM of Linux 3.1 kernels or Windows
>> 2008+.
> 
> +1 to a new MPM on trunk. This gives it more time to settle and to stabilize
> without disrupting current stuff. And if it is fast and stable it will 
> certainly
> cause the 'older' MPM to drop in userbase :-).
> IMHO this would even open a path to 2.4.x provided that we do not need any 
> other
> non backportable changes to do this.
> 
> Regards
> 
> Rüdiger
> 

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Yann Ylavic]

(one per
bucket, although muliple children will use the same listener at the same
time should num_buckets > num_listeners).

That's what I mean by "each child is dedicated" (without SO_REUSEPORT),
each will accept connections from a single listening socket only.
 Is that correct?

If so, this is a change with regard to the current prefork sizing habits.
Currently with prefork (but this is also true for other threaded MPMs
modulo ThreadsPerChild), when the admin sizes httpd.conf (max
children/clients/... according to the hardware capabilites, applications
needs...), (s)he expects each child process to handle all the incoming
connections (on any listening socket).
Should one VirtualHost (on one listening socket) handle more traffic than
the others, its load is distributed on all the children.
The admin does not have to worry about how much process
es
for this listening socket or that other.

With this patch though, it no longer holds, there can be idle processes (no
activity on their listener) while others are busy (and even full).
It's worth it if the load is closed for all the listeners, but that won't
fit all the existing configurations...
Hence I think we need a way to configure this.

Regards,
Yann.



>
>
> This is our first patch to the open source and Apache community. We are
> still on the learning curve about a lot of things. Your feedback and
> comments really help us!
>
>
>
> Please let me know if you have any further questions.
>
>
>
> Thanks,
>
> Yingqi
>
>
>
>
>
> *From:* Yann Ylavic [mailto:ylavic@gmail.com]
> *Sent:* Wednesday, March 05, 2014 5:04 AM
> *To:* httpd
> *Subject:* Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
> SO_REUSEPORT support
>
>
>
> Hi Yingqi,
>
>
>
> I'm a bit confused about the patch, mainly because it seems to handle the
> same way both with and without SO_REUSEPORT available, while SO_REUSEPORT
> could (IMHO) be handled in children only (a less intrusive way).
>
> With SO_REUSEPORT, I would have expected the accept mutex to be useless
> since, if I understand correcly the option, multiple processes/threads can
> accept() simultaneously provided they use their own socket (each one
> bound/listening on the same addr:port).
>
> Couldn't then each child duplicate the listeners (ie. new
> socket+bind(SO_REUSEPORT)+listen), before switching UIDs, and then poll()
> all of them without synchronisation (accept() is probably not an option for
> timeout reasons), and then get fair scheduling from the OS (for all the
> listeners)?
>
> Is the lock still needed because the duplicated listeners are inherited
> from the parent process?
>
>
>
> Without SO_REUSEPORT, if I understand correctly still, each child will
> poll() a single listener to avoid the serialized accept.
>
> On the other hand, each child is dedicated, won't one have to multiply the
> configured ServerLimit by the number of Listen to achieve the same (maximum
> theoretical) scalability with regard to all the listeners?
>
> I don't pretend it is a good or bad thing, just figuring out what could
> then be a "rule" to size the configuration (eg.
> MaxClients/ServerLimit/#cores/#Listen).
>
> It seems to me that the patches with and without SO_REUSEPORT should be
> separate ones, but I may be missing something.
>
> Also, but this is not related to this patch particularly (addressed to
> who knows), it's unclear to me why an accept mutex is needed at all.
>
> Multiple processes poll()ing the same inherited socket is safe but not
> multiple ones? Is that an OS issue? Process wide only? Still (in)valid in
> latest OSes?
>
>
>
> Thanks for the patch anyway, it looks promising.
>
> Regards,
>
> Yann.
>
>
>
> On Sat, Jan 25, 2014 at 12:25 AM, Lu, Yingqi  wrote:
>
> Dear All,
>
>
>
> Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel
> Xeon 2600 series systems, using an open source three tier social networking
> web server workload, revealed performance scaling issues.  In current
> software single listen statement (listen 80) provides better scalability
> due to un-serialized accept. However, when system is under very high load,
> this can lead to big number of child processes stuck in D state.
>
>
>
>
>  On the other hand, the serialized accept approach cannot scale with the
> high load either.  In our analysis, a 32-thread system, with 2 listen
> statements specified, could scale to just 70% utilization, and a 64-thread
> system, with signal listen statement specified (listen 80, 4 network
> interfaces), could scale to only 60% utilization.
>
>
>
> Based on those findings, we created a prototype patch for prefork mpm

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Graham Leggett]

On 06 Mar 2014, at 10:15 AM, "Plüm, Rüdiger, Vodafone Group" 
 wrote:

> +1 to a new MPM on trunk. This gives it more time to settle and to stabilize
> without disrupting current stuff. And if it is fast and stable it will 
> certainly
> cause the 'older' MPM to drop in userbase :-).
> IMHO this would even open a path to 2.4.x provided that we do not need any 
> other
> non backportable changes to do this.

+1.

Regards,
Graham
--

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Plüm , Rüdiger , Vodafone Group]

> -Original Message-
> From: William A. Rowe Jr. [mailto:wmr...@gmail.com]
> Sent: Donnerstag, 6. März 2014 06:58
> To: dev@httpd.apache.org
> Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with
> SO_REUSEPORT support
> 
> 
> If you want to truly re-architect the MPM, by all means, propose it as
> another MPM module.  If it isn't adopted here, please don't hesitate
> to offer it to interested users as separate source (although I hope we
> find a way to adopt it.)
> 
> The idea of different MPM's was that they were swappable.  MPM foo
> isn't MPM bar.  E.g., worker, prefork, event each have their own tree.
>  Likewise, there is nothing stopping us from having 2, or 3 MPM's on
> Windows, and there is nothing stopping us from stating that there is a
> prerequisite on a particular MPM of Linux 3.1 kernels or Windows
> 2008+.

+1 to a new MPM on trunk. This gives it more time to settle and to stabilize
without disrupting current stuff. And if it is fast and stable it will certainly
cause the 'older' MPM to drop in userbase :-).
IMHO this would even open a path to 2.4.x provided that we do not need any other
non backportable changes to do this.

Regards

Rüdiger

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Bill,

Thanks very much for your email and I am really happy that I got lots of very 
good feedbacks on the email list.

The patch was created only for Linux Prefork mpm so that it should not impact 
winnt_mpm. I may misunderstand you here, but do you mean in order to adopt the 
patch, we need to extend it for winnt_mpm?

Regarding to the testing result, what we provided was based on RHEL 6.2 (server 
version) with kernel 3.10.4. We measured the throughput as operations/sec as 
well as the response time defined by the time that a request sending from the 
client till it gets the response back. It is a three tier webserver workload. 
We measured the throughput on the frontend webserver tier (Apache httpd with 
Prefork + PHP as libphp5.so under httpd/modules).

Thanks,
Yingqi 

-Original Message-
From: William A. Rowe Jr. [mailto:wmr...@gmail.com] 
Sent: Wednesday, March 05, 2014 9:58 PM
To: dev@httpd.apache.org
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Yingqi,

as one of the 'Windows folks' here, your idea is very intriguing, and I'm sorry 
that other issues have distracted me from giving it the attention it deserves.

If you want to truly re-architect the MPM, by all means, propose it as another 
MPM module.  If it isn't adopted here, please don't hesitate to offer it to 
interested users as separate source (although I hope we find a way to adopt it.)

The idea of different MPM's was that they were swappable.  MPM foo isn't MPM 
bar.  E.g., worker, prefork, event each have their own tree.
 Likewise, there is nothing stopping us from having 2, or 3 MPM's on Windows, 
and there is nothing stopping us from stating that there is a prerequisite on a 
particular MPM of Linux 3.1 kernels or Windows
2008+.

The Windows build system hasn't been so flexible, but this can be remediated 
with cmake, as folks have spent many hours to accomplish.
I understand you are probably relying on functions authored entirely for the 
winnt_mpm, and we can re-factor those on trunk out to the os/win32/ directory 
so that MPM's may share them.

The definition of the word "prefork" is a single thread process which handles a 
request.  Please don't misuse the phrase, and without reviewing your code, I'll 
presume that is what you meant.

I don't doubt your results of benchmarking, but please make note that only 
Windows Server OS's can actually be used to perform any benchmarks.  Any 
'desktop' enterprise, professional or home editions are deliberately hobbled, 
and IMHO the project should make no accommodation for vendor stupidity.

In terms of benchmarking, I don't know how you measured, but if you can peg a 
machine at 95% total utilization yet httpd shows itself consuming only 70% or 
60%, that means it is kernel-bound.  That is usually a good thing, that the app 
is operating optimally and is only constrained by the architecture.

I think I understand where you are going with reuseport.  That doesn't equate 
to the Unix OS's... they can distribute the already opened listener to an 
unlimited number of forks.  On windows, we also distribute the listener through 
a write/stdin channel to the child process.  What doesn't work well is for 
parallel windows children to share certain resources such as the error log, 
access log etc.  But we can contend with that issue.  What we can't contend 
with is what 3rd party modules have chosen to do, and almost any patch you 
offer is not going to be suitable for binary compatibility with 3rd party httpd 
2.4 modules compiled for windows, so your patch presented for the 2.4 branch is 
rejected.

That said, we should endeavor to solve this for 2.6 (or 3.0 or whatever we call 
the 'next httpd').  We are all out of fresh ideas, so proposals such as yours 
are a welcome sight!!!

Finally, please do have patience, large patches require time for us to digest, 
and we have limited amounts of that resource.  As I mention, adding a whole new 
MPM directory to trunk, alone, should meet very little resistance for any 
architectures.

Thank you for your posts, and please do not feel ignored.  There are a handful 
of people active and we all have many details to attend to.

Yours,

Bill

On Fri, Jan 24, 2014 at 5:25 PM, Lu, Yingqi  wrote:
> Dear All,
>
>
>
> Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread 
> Intel Xeon 2600 series systems, using an open source three tier social 
> networking web server workload, revealed performance scaling issues.  
> In current software single listen statement (listen 80) provides 
> better scalability due to un-serialized accept. However, when system 
> is under very high load, this can lead to big number of child 
> processes stuck in D state. On the other hand, the serialized accept approach 
> cannot scale with the high load either.

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Arkadiusz Miśkiewicz]

On Thursday 06 of March 2014, William A. Rowe Jr. wrote:

> If you want to truly re-architect the MPM, by all means, propose it as
> another MPM module.  If it isn't adopted here, please don't hesitate
> to offer it to interested users as separate source (although I hope we
> find a way to adopt it.)
> 
> The idea of different MPM's was that they were swappable.  MPM foo
> isn't MPM bar.  E.g., worker, prefork, event each have their own tree.
>  Likewise, there is nothing stopping us from having 2, or 3 MPM's on
> Windows, and there is nothing stopping us from stating that there is a
> prerequisite on a particular MPM of Linux 3.1 kernels or Windows
> 2008+.

I dislike idea of "yet another mpm". More mpm means that each mpm gets lower 
developer resources and lower testing (and external mpm, distributed outside 
apache get almost no devs and no testing).

Less MPMs is better IMO. So better to improve existing ones than invent new 
one.

> (although I hope we find a way to adopt it.)

+1

-- 
Arkadiusz Miśkiewicz, arekm / maven.pl

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[William A. Rowe Jr.]

Yingqi,

as one of the 'Windows folks' here, your idea is very intriguing, and
I'm sorry that other issues have distracted me from giving it the
attention it deserves.

If you want to truly re-architect the MPM, by all means, propose it as
another MPM module.  If it isn't adopted here, please don't hesitate
to offer it to interested users as separate source (although I hope we
find a way to adopt it.)

The idea of different MPM's was that they were swappable.  MPM foo
isn't MPM bar.  E.g., worker, prefork, event each have their own tree.
 Likewise, there is nothing stopping us from having 2, or 3 MPM's on
Windows, and there is nothing stopping us from stating that there is a
prerequisite on a particular MPM of Linux 3.1 kernels or Windows
2008+.

The Windows build system hasn't been so flexible, but this can be
remediated with cmake, as folks have spent many hours to accomplish.
I understand you are probably relying on functions authored entirely
for the winnt_mpm, and we can re-factor those on trunk out to the
os/win32/ directory so that MPM's may share them.

The definition of the word "prefork" is a single thread process which
handles a request.  Please don't misuse the phrase, and without
reviewing your code, I'll presume that is what you meant.

I don't doubt your results of benchmarking, but please make note that
only Windows Server OS's can actually be used to perform any
benchmarks.  Any 'desktop' enterprise, professional or home editions
are deliberately hobbled, and IMHO the project should make no
accommodation for vendor stupidity.

In terms of benchmarking, I don't know how you measured, but if you
can peg a machine at 95% total utilization yet httpd shows itself
consuming only 70% or 60%, that means it is kernel-bound.  That is
usually a good thing, that the app is operating optimally and is only
constrained by the architecture.

I think I understand where you are going with reuseport.  That doesn't
equate to the Unix OS's... they can distribute the already opened
listener to an unlimited number of forks.  On windows, we also
distribute the listener through a write/stdin channel to the child
process.  What doesn't work well is for parallel windows children to
share certain resources such as the error log, access log etc.  But we
can contend with that issue.  What we can't contend with is what 3rd
party modules have chosen to do, and almost any patch you offer is not
going to be suitable for binary compatibility with 3rd party httpd 2.4
modules compiled for windows, so your patch presented for the 2.4
branch is rejected.

That said, we should endeavor to solve this for 2.6 (or 3.0 or
whatever we call the 'next httpd').  We are all out of fresh ideas, so
proposals such as yours are a welcome sight!!!

Finally, please do have patience, large patches require time for us to
digest, and we have limited amounts of that resource.  As I mention,
adding a whole new MPM directory to trunk, alone, should meet very
little resistance for any architectures.

Thank you for your posts, and please do not feel ignored.  There are a
handful of people active and we all have many details to attend to.

Yours,

Bill

On Fri, Jan 24, 2014 at 5:25 PM, Lu, Yingqi  wrote:
> Dear All,
>
>
>
> Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel
> Xeon 2600 series systems, using an open source three tier social networking
> web server workload, revealed performance scaling issues.  In current
> software single listen statement (listen 80) provides better scalability due
> to un-serialized accept. However, when system is under very high load, this
> can lead to big number of child processes stuck in D state. On the other
> hand, the serialized accept approach cannot scale with the high load either.
> In our analysis, a 32-thread system, with 2 listen statements specified,
> could scale to just 70% utilization, and a 64-thread system, with signal
> listen statement specified (listen 80, 4 network interfaces), could scale to
> only 60% utilization.
>
>
>
> Based on those findings, we created a prototype patch for prefork mpm which
> extends performance and thread utilization. In Linux kernel newer than 3.9,
> SO_REUSEPORT is enabled. This feature allows multiple sockets listen to the
> same IP:port and automatically round robins connections. We use this feature
> to create multiple duplicated listener records of the original one and
> partition the child processes into buckets. Each bucket listens to 1
> IP:port. In case of old kernel which does not have the SO_REUSEPORT enabled,
> we modified the "multiple listen statement case" by creating 1 listen record
> for each listen statement and partitioning the child processes into
> different buckets. Each bucket listens to 1 IP:port.
>
>
>
> Quick tests of the patch, running the same workload, demonstrated a 22%
> throughput increase with 32-threads system and 2 listen statements (Linux
> kernel 3.10.4). With the older kernel (Linux Kernel 3.8.8, without
> S

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi Yann,

Thanks very much for your email.

1. If I understand correctly (please correct me if not), do you suggest 
duplicating the listen socks inside the child process with SO_REUSEPROT 
enabled? Yes, I agree this would be a cleaner implementation and I actually 
tried that before. However, I encountered the "connection reset" error since 
the number of the child process is changing. I googled online and found it 
actually being discussed here at http://lwn.net/Articles/542629/.

2. Then, I decided to do the socket duplication in the parent process. The goal 
of this change is to extend the CPU thread scalability with the big thread 
count system. Therefore, I just very simply defined 
number_of_listen_buckets=total_number_active_thread/8, and each listen bucket 
has a dedicated listener. I do not want to over duplicate the socket; 
otherwise, it would create too many child processes at the beginning. One 
listen bucket should have at least one child process to start with. However, 
this is only my understanding and it may not be correct and complete. If you 
have other ideas, please share with us. Feedbacks and comments are very welcome 
here :)

3. I am struggling with myself as well on if we should put with and without 
SO_REUSEPORT into two different patches. The only reason I put them together is 
because they both use the concept of listen buckets. If you think it would make 
more sense to separate them into two patches, I can certainly do that. Also, I 
am a little bit confused about your comments "On the other hand, each child is 
dedicated, won't one have to multiply the configured ServerLimit by the number 
of Listen to achieve the same (maximum theoretical) scalability with regard to 
all the listeners?". Can you please explain a little bit more on this? Really 
appreciate.

This is our first patch to the open source and Apache community. We are still 
on the learning curve about a lot of things. Your feedback and comments really 
help us!

Please let me know if you have any further questions.

Thanks,
Yingqi


From: Yann Ylavic [mailto:ylavic@gmail.com]
Sent: Wednesday, March 05, 2014 5:04 AM
To: httpd
Subject: Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi Yingqi,

I'm a bit confused about the patch, mainly because it seems to handle the same 
way both with and without SO_REUSEPORT available, while SO_REUSEPORT could 
(IMHO) be handled in children only (a less intrusive way).
With SO_REUSEPORT, I would have expected the accept mutex to be useless since, 
if I understand correcly the option, multiple processes/threads can accept() 
simultaneously provided they use their own socket (each one bound/listening on 
the same addr:port).
Couldn't then each child duplicate the listeners (ie. new 
socket+bind(SO_REUSEPORT)+listen), before switching UIDs, and then poll() all 
of them without synchronisation (accept() is probably not an option for timeout 
reasons), and then get fair scheduling from the OS (for all the listeners)?
Is the lock still needed because the duplicated listeners are inherited from 
the parent process?

Without SO_REUSEPORT, if I understand correctly still, each child will poll() a 
single listener to avoid the serialized accept.
On the other hand, each child is dedicated, won't one have to multiply the 
configured ServerLimit by the number of Listen to achieve the same (maximum 
theoretical) scalability with regard to all the listeners?
I don't pretend it is a good or bad thing, just figuring out what could then be 
a "rule" to size the configuration (eg. MaxClients/ServerLimit/#cores/#Listen).
It seems to me that the patches with and without SO_REUSEPORT should be 
separate ones, but I may be missing something.
Also, but this is not related to this patch particularly (addressed to who 
knows), it's unclear to me why an accept mutex is needed at all.
Multiple processes poll()ing the same inherited socket is safe but not multiple 
ones? Is that an OS issue? Process wide only? Still (in)valid in latest OSes?

Thanks for the patch anyway, it looks promising.
Regards,
Yann.

On Sat, Jan 25, 2014 at 12:25 AM, Lu, Yingqi 
mailto:yingqi...@intel.com>> wrote:
Dear All,

Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel Xeon 
2600 series systems, using an open source three tier social networking web 
server workload, revealed performance scaling issues.  In current software 
single listen statement (listen 80) provides better scalability due to 
un-serialized accept. However, when system is under very high load, this can 
lead to big number of child processes stuck in D state.


On the other hand, the serialized accept approach cannot scale with the high 
load either.  In our analysis, a 32-thread system, with 2 listen statements 
specified, could scale to just 70% utilization, and a 64-thread system, with 
signal listen statement specified (listen 80, 4 ne

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Yann Ylavic]

On Wed, Mar 5, 2014 at 2:04 PM, Yann Ylavic  wrote:

> Also, but this is not related to this patch particularly (addressed to
> who knows), it's unclear to me why an accept mutex is needed at all.
> Multiple processes poll()ing the same inherited socket is safe but not
> multiple ones? Is that an OS issue? Process wide only? Still (in)valid in
> latest OSes?
>

I mean
 when 
SINGLE_LISTEN_UNSERIALIZED_ACCEPT is
set only, the OS has the capability to do unserialized accept with one
socket but not any more.

Re: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Yann Ylavic]

Hi Yingqi,

I'm a bit confused about the patch, mainly because it seems to handle the
same way both with and without SO_REUSEPORT available, while SO_REUSEPORT
could (IMHO) be handled in children only (a less intrusive way).

With SO_REUSEPORT, I would have expected the accept mutex to be useless
since, if I understand correcly the option, multiple processes/threads can
accept() simultaneously provided they use their own socket (each one
bound/listening on the same addr:port).
Couldn't then each child duplicate the listeners (ie. new
socket+bind(SO_REUSEPORT)+listen), before switching UIDs, and then poll()
all of them without synchronisation (accept() is probably not an option for
timeout reasons), and then get fair scheduling from the OS (for all the
listeners)?
Is the lock still needed because the duplicated listeners are inherited
from the parent process?

Without SO_REUSEPORT, if I understand correctly still, each child will
poll() a single listener to avoid the serialized accept.
On the other hand, each child is dedicated, won't one have to multiply the
configured ServerLimit by the number of Listen to achieve the same (maximum
theoretical) scalability with regard to all the listeners?
I don't pretend it is a good or bad thing, just figuring out what could
then be a "rule" to size the configuration (eg.
MaxClients/ServerLimit/#cores/#Listen).

It seems to me that the patches with and without SO_REUSEPORT should be
separate ones, but I may be missing something.

Also, but this is not related to this patch particularly (addressed to who
knows), it's unclear to me why an accept mutex is needed at all.
Multiple processes poll()ing the same inherited socket is safe but not
multiple ones? Is that an OS issue? Process wide only? Still (in)valid in
latest OSes?

Thanks for the patch anyway, it looks promising.

Regards,
Yann.

On Sat, Jan 25, 2014 at 12:25 AM, Lu, Yingqi  wrote:

>  Dear All,
> 
>
>
>
> Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel
> Xeon 2600 series systems, using an open source three tier social networking
> web server workload, revealed performance scaling issues.  In current
> software single listen statement (listen 80) provides better scalability
> due to un-serialized accept. However, when system is under very high load,
> this can lead to big number of child processes stuck in D state.
>



> On the other hand, the serialized accept approach cannot scale with the
> high load either.  In our analysis, a 32-thread system, with 2 listen
> statements specified, could scale to just 70% utilization, and a 64-thread
> system, with signal listen statement specified (listen 80, 4 network
> interfaces), could scale to only 60% utilization.
>
>
>
> Based on those findings, we created a prototype patch for prefork mpm
> which extends performance and thread utilization. In Linux kernel newer
> than 3.9, SO_REUSEPORT is enabled. This feature allows multiple sockets
> listen to the same IP:port and automatically round robins connections. We
> use this feature to create multiple duplicated listener records of the
> original one and partition the child processes into buckets. Each bucket
> listens to 1 IP:port. In case of old kernel which does not have the
> SO_REUSEPORT enabled, we modified the "multiple listen statement case" by
> creating 1 listen record for each listen statement and partitioning the
> child processes into different buckets. Each bucket listens to 1 IP:port.
>
>
>
> Quick tests of the patch, running the same workload, demonstrated a 22%
> throughput increase with 32-threads system and 2 listen statements (Linux
> kernel 3.10.4). With the older kernel (Linux Kernel 3.8.8, without
> SO_REUSEPORT), 10% performance gain was measured. With single listen
> statement (listen 80) configuration, we observed over 2X performance
> improvements on modern dual socket Intel platforms (Linux Kernel 3.10.4).
> We also observed big reduction in response time, in addition to the
> throughput improvement gained in our tests 1.
>
>
>
> Following the feedback from the bugzilla website where we originally
> submitted the patch, we removed the dependency of APR change to simplify
> the patch testing process. Thanks Jeff Trawick for his good suggestion! We
> are also actively working on extending the patch to worker and event MPMs,
> as a next step. Meanwhile, we would like to gather comments from all of you
> on the current prefork patch. Please take some time test it and let us know
> how it works in your environment.
>
>
>
> This is our first patch to the Apache community. Please help us review it
> and let us know if there is anything we might revise to improve it. Your
> feedback is very much appreciated.
>
>
>
> *Configuration:*
>
> 
>
> ListenBacklog 105384
>
> ServerLimit 105000
>
> MaxClients 1024
>
> MaxRequestsPerChild 0
>
> StartServers 64
>
> MinSpareServers 8
>
> MaxSpareServers 16
>
> 
>
>
>
> 1. Software and workloads used in performa

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

I am reattaching the patch in case you missed the original email.

Thanks,
Yingqi

From: Lu, Yingqi
Sent: Monday, February 10, 2014 11:09 AM
To: dev@httpd.apache.org
Subject: RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT 
support

Hi All,

I just want to ping again on this patch to see if there are any feedback and 
comments. This is our first patch to the Apache community. Please let us know 
if there is anything we can do to help you test and comment the patch.

Thanks,
Yingqi

From: Lu, Yingqi
Sent: Friday, January 24, 2014 3:26 PM
To: dev@httpd.apache.org<mailto:dev@httpd.apache.org>
Subject: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

Dear All,

Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel Xeon 
2600 series systems, using an open source three tier social networking web 
server workload, revealed performance scaling issues.  In current software 
single listen statement (listen 80) provides better scalability due to 
un-serialized accept. However, when system is under very high load, this can 
lead to big number of child processes stuck in D state. On the other hand, the 
serialized accept approach cannot scale with the high load either.  In our 
analysis, a 32-thread system, with 2 listen statements specified, could scale 
to just 70% utilization, and a 64-thread system, with signal listen statement 
specified (listen 80, 4 network interfaces), could scale to only 60% 
utilization.

Based on those findings, we created a prototype patch for prefork mpm which 
extends performance and thread utilization. In Linux kernel newer than 3.9, 
SO_REUSEPORT is enabled. This feature allows multiple sockets listen to the 
same IP:port and automatically round robins connections. We use this feature to 
create multiple duplicated listener records of the original one and partition 
the child processes into buckets. Each bucket listens to 1 IP:port. In case of 
old kernel which does not have the SO_REUSEPORT enabled, we modified the 
"multiple listen statement case" by creating 1 listen record for each listen 
statement and partitioning the child processes into different buckets. Each 
bucket listens to 1 IP:port.

Quick tests of the patch, running the same workload, demonstrated a 22% 
throughput increase with 32-threads system and 2 listen statements (Linux 
kernel 3.10.4). With the older kernel (Linux Kernel 3.8.8, without 
SO_REUSEPORT), 10% performance gain was measured. With single listen statement 
(listen 80) configuration, we observed over 2X performance improvements on 
modern dual socket Intel platforms (Linux Kernel 3.10.4). We also observed big 
reduction in response time, in addition to the throughput improvement gained in 
our tests 1.

Following the feedback from the bugzilla website where we originally submitted 
the patch, we removed the dependency of APR change to simplify the patch 
testing process. Thanks Jeff Trawick for his good suggestion! We are also 
actively working on extending the patch to worker and event MPMs, as a next 
step. Meanwhile, we would like to gather comments from all of you on the 
current prefork patch. Please take some time test it and let us know how it 
works in your environment.

This is our first patch to the Apache community. Please help us review it and 
let us know if there is anything we might revise to improve it. Your feedback 
is very much appreciated.

Configuration:

ListenBacklog 105384
ServerLimit 105000
MaxClients 1024
MaxRequestsPerChild 0
StartServers 64
MinSpareServers 8
MaxSpareServers 16


1. Software and workloads used in performance tests may have been optimized for 
performance only on Intel microprocessors. Performance tests, such as SYSmark 
and MobileMark, are measured using specific computer systems, components, 
software, operations and functions. Any change to any of those factors may 
cause the results to vary. You should consult other information and performance 
tests to assist you in fully evaluating your contemplated purchases, including 
the performance of that product when combined with other products.

Thanks,
Yingqi


unified.diff.httpd-2.4.7.patch
Description: unified.diff.httpd-2.4.7.patch

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi All,

I just want to ping again on this patch to see if there are any feedback and 
comments. This is our first patch to the Apache community. Please let us know 
if there is anything we can do to help you test and comment the patch.

Thanks,
Yingqi

From: Lu, Yingqi
Sent: Friday, January 24, 2014 3:26 PM
To: dev@httpd.apache.org
Subject: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

Dear All,

Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel Xeon 
2600 series systems, using an open source three tier social networking web 
server workload, revealed performance scaling issues.  In current software 
single listen statement (listen 80) provides better scalability due to 
un-serialized accept. However, when system is under very high load, this can 
lead to big number of child processes stuck in D state. On the other hand, the 
serialized accept approach cannot scale with the high load either.  In our 
analysis, a 32-thread system, with 2 listen statements specified, could scale 
to just 70% utilization, and a 64-thread system, with signal listen statement 
specified (listen 80, 4 network interfaces), could scale to only 60% 
utilization.

Based on those findings, we created a prototype patch for prefork mpm which 
extends performance and thread utilization. In Linux kernel newer than 3.9, 
SO_REUSEPORT is enabled. This feature allows multiple sockets listen to the 
same IP:port and automatically round robins connections. We use this feature to 
create multiple duplicated listener records of the original one and partition 
the child processes into buckets. Each bucket listens to 1 IP:port. In case of 
old kernel which does not have the SO_REUSEPORT enabled, we modified the 
"multiple listen statement case" by creating 1 listen record for each listen 
statement and partitioning the child processes into different buckets. Each 
bucket listens to 1 IP:port.

Quick tests of the patch, running the same workload, demonstrated a 22% 
throughput increase with 32-threads system and 2 listen statements (Linux 
kernel 3.10.4). With the older kernel (Linux Kernel 3.8.8, without 
SO_REUSEPORT), 10% performance gain was measured. With single listen statement 
(listen 80) configuration, we observed over 2X performance improvements on 
modern dual socket Intel platforms (Linux Kernel 3.10.4). We also observed big 
reduction in response time, in addition to the throughput improvement gained in 
our tests 1.

Following the feedback from the bugzilla website where we originally submitted 
the patch, we removed the dependency of APR change to simplify the patch 
testing process. Thanks Jeff Trawick for his good suggestion! We are also 
actively working on extending the patch to worker and event MPMs, as a next 
step. Meanwhile, we would like to gather comments from all of you on the 
current prefork patch. Please take some time test it and let us know how it 
works in your environment.

This is our first patch to the Apache community. Please help us review it and 
let us know if there is anything we might revise to improve it. Your feedback 
is very much appreciated.

Configuration:

ListenBacklog 105384
ServerLimit 105000
MaxClients 1024
MaxRequestsPerChild 0
StartServers 64
MinSpareServers 8
MaxSpareServers 16


1. Software and workloads used in performance tests may have been optimized for 
performance only on Intel microprocessors. Performance tests, such as SYSmark 
and MobileMark, are measured using specific computer systems, components, 
software, operations and functions. Any change to any of those factors may 
cause the results to vary. You should consult other information and performance 
tests to assist you in fully evaluating your contemplated purchases, including 
the performance of that product when combined with other products.

Thanks,
Yingqi

RE: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

[Lu, Yingqi]

Hi All,

I just want to check if there are feedback/comments to this patch?

Thanks,
Yingqi

From: Lu, Yingqi
Sent: Friday, January 24, 2014 3:26 PM
To: dev@httpd.apache.org
Subject: [PATCH ASF bugzilla# 55897]prefork_mpm patch with SO_REUSEPORT support

Dear All,

Our analysis of Apache httpd 2.4.7 prefork mpm, on 32 and 64 thread Intel Xeon 
2600 series systems, using an open source three tier social networking web 
server workload, revealed performance scaling issues.  In current software 
single listen statement (listen 80) provides better scalability due to 
un-serialized accept. However, when system is under very high load, this can 
lead to big number of child processes stuck in D state. On the other hand, the 
serialized accept approach cannot scale with the high load either.  In our 
analysis, a 32-thread system, with 2 listen statements specified, could scale 
to just 70% utilization, and a 64-thread system, with signal listen statement 
specified (listen 80, 4 network interfaces), could scale to only 60% 
utilization.

Based on those findings, we created a prototype patch for prefork mpm which 
extends performance and thread utilization. In Linux kernel newer than 3.9, 
SO_REUSEPORT is enabled. This feature allows multiple sockets listen to the 
same IP:port and automatically round robins connections. We use this feature to 
create multiple duplicated listener records of the original one and partition 
the child processes into buckets. Each bucket listens to 1 IP:port. In case of 
old kernel which does not have the SO_REUSEPORT enabled, we modified the 
"multiple listen statement case" by creating 1 listen record for each listen 
statement and partitioning the child processes into different buckets. Each 
bucket listens to 1 IP:port.

Quick tests of the patch, running the same workload, demonstrated a 22% 
throughput increase with 32-threads system and 2 listen statements (Linux 
kernel 3.10.4). With the older kernel (Linux Kernel 3.8.8, without 
SO_REUSEPORT), 10% performance gain was measured. With single listen statement 
(listen 80) configuration, we observed over 2X performance improvements on 
modern dual socket Intel platforms (Linux Kernel 3.10.4). We also observed big 
reduction in response time, in addition to the throughput improvement gained in 
our tests 1.

Following the feedback from the bugzilla website where we originally submitted 
the patch, we removed the dependency of APR change to simplify the patch 
testing process. Thanks Jeff Trawick for his good suggestion! We are also 
actively working on extending the patch to worker and event MPMs, as a next 
step. Meanwhile, we would like to gather comments from all of you on the 
current prefork patch. Please take some time test it and let us know how it 
works in your environment.

This is our first patch to the Apache community. Please help us review it and 
let us know if there is anything we might revise to improve it. Your feedback 
is very much appreciated.

Configuration:

ListenBacklog 105384
ServerLimit 105000
MaxClients 1024
MaxRequestsPerChild 0
StartServers 64
MinSpareServers 8
MaxSpareServers 16


1. Software and workloads used in performance tests may have been optimized for 
performance only on Intel microprocessors. Performance tests, such as SYSmark 
and MobileMark, are measured using specific computer systems, components, 
software, operations and functions. Any change to any of those factors may 
cause the results to vary. You should consult other information and performance 
tests to assist you in fully evaluating your contemplated purchases, including 
the performance of that product when combined with other products.

Thanks,
Yingqi