AW: mod_proxy_fcgi and flush

[Plüm , Rüdiger , Vodafone Group]

Von: Luca Toscano [mailto:toscano.l...@gmail.com]
Gesendet: Samstag, 8. Juli 2017 09:52
An: Apache HTTP Server Development List 
Betreff: Re: mod_proxy_fcgi and flush

Hi Jacob, Helmut!

2017-07-06 20:54 GMT+02:00 Jacob Champion 
>:
On 07/06/2017 11:13 AM, Jim Jagielski wrote:
works 4 me...

Doesn't for me. E.g. with a script like



it takes 1 second to receive a single chunk with both lines in it.

From a quick skim I assume this is because we don't use nonblocking sockets in 
the proxy implementation. (There's even a note in mod_proxy_fcgi that says, 
"Yes it sucks to [get the actual data] in a second recv call, this will 
eventually change when we move to real nonblocking recv calls.")

Quick check from my side too, so not sure if it makes sense. IIUC the comment 
is about getting all the headers from the FCGI backend (get_data_full(..., 
AP_FCGI_HEADER_LEN)), then get the response body in another one (the [actual 
data]).

I checked mod_fcgi as Helmut suggested and it seems to me that the -flush 
feature is a simple "flush every data that you receive", so I tested the 
following patch with Jacob's php example code and it seems doing what Helmut 
asked (please correct me if I am wrong).

Caveat: I had to set output_buffering = Off in my php-fpm's php.ini config file.

http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi-force_flush.patch

Flushing unconditionally is a bad idea performance wise. Please have a look how 
ajp solved this issue:

https://svn.apache.org/r327185
https://svn.apache.org/r384580
https://svn.apache.org/r390210
https://svn.apache.org/viewvc/httpd/httpd/trunk/modules/proxy/mod_proxy_ajp.c?r1=325879=390210

Hint: The above diff contains further unrelated changes.

Regards

Rüdiger

Re: An ask for eyes on proposal

[William A Rowe Jr]

Based on the fact that Jim's advanced this for consideration for 2.4.28,
any further feedback on the following proposal to make RemoteIPProxyProtocol
directive into a whitelist (to compliment the current blacklist directive)? E.g.
in the spirit of the protocol, assign specific remote ip consumers to the list
of those where we accept PROXY protocol wrapping?


On Fri, Jun 9, 2017 at 8:29 AM, William A Rowe Jr  wrote:
> On Fri, Jun 9, 2017 at 4:17 AM, Sander Hoentjen  wrote:
>> On 06/08/2017 07:30 PM, Daniel Ruggeri wrote:
>>> Hi, all;
>>> With the proposal to T set for Monday, I wanted to draw attention to
>>> the PROXY protocol proposal in STATUS. Just hoping for a quick review.
>>> I know it appears to be a large change, but as I worked through the
>>> feedback, ten of the commits effectively got coded out. What we are
>>> left with is essentially just the donated code + safety around IPv6 +
>>> the ability to designate subnets that do not get PROXY processing.
>>
>> [...] I still believe it would be better to specify enabling
>> Proxy Protocol on a server, not vhost level. Because well, once you
>> enable it in one vhost it gets enabled for all vhosts using that port/ip
>> combination.
>>
>> Here is what I said before about it:
>>
>> Right now the patch proposes RemoteIPProxyProtocol inside a vhost config, 
>> but wouldn't it be better (since it is connection-specific) to have 
>> something like a ProxyProtocolListen directive? Where you say instead of:
>> --
>> 
>> RemoteIPProxyProtocol On
>> 
>> --
>> Something like:
>> --
>> ProxyProtocolListen 127.0.0.1:9001
>> or
>> ProxyProtocolEnable 127.0.0.1:9001
>> --
>>
>> IMHO this is much cleaner than within a vhost (because that has side-effects 
>> on other vhosts as well)
>
> As this lives in mod_remoteip (for better or worse) let's look at what
> context mod_remoteip is configured in; we set up a list of those local
> or global *client* IP's which we trust to provide legit x-f-f (or remote-ip
> or otherly named) true IP address header fields.
>
> in the PROXY protocol case, we configure which *client* IP's which
> we *require* to submit a PROXY protocol line. Right now, we do that
> as a RemoteIPProxyProtocolExceptions list of those which we do *not*
> allow to submit a PROXY protocol line. I proposed we make the config
> simpler, in theory, by listing those we will trust.
>
> To your example, the *global* config line;
>
> RemoteIPProxyProtocol 127.0.0.1  [or 127.0.0.0/24]
>
> would configure all locally routed *client* requests, irrespective of
> which by-IP vhost, to require the PROXY protocol line. Requests
> from other hosts would be denied.
>
> I think that's sufficient. But if we wanted to implement your basic
> idea, we would still have the complication that we need to infer
> whether 9001 is a http, https, or h2 listener following the PROXY
> line processing. Your proposed syntax didn't really touch on that.
>
> It is still possible to override behavior by-vhost (ip-based, we are
> unprepared to read the TLS SNI or Host header at that point)
> but I don't see any application to do so. A given client is either
> an haproxy or similar, or it is not.

Re: httpd 2.4.26 with apr 1.6 ExtFilterDefine

[William A Rowe Jr]

I am very confused, examined apr 1.6 threadproc deltas, now looking at the
mpm, but reporter isolated to apr. Next place I plan to look is is the
fileio abstraction.

I am beginning to suspect this is 'that customer'... In my experience, the
one who duct tapes and binds in entirely unrelated shared libs on Windows
and hopes they will work. If they are passing around apr_os_ objects, they
simply won't when the C Runtime flavors differ.

But, that isn't the entire problem, an executed external process should be
independent of the parent process Runtime. Unless a wrong msvcrt.dll is
wedged in the PATH.

Steffen this one sounds like something your user community could refute or
confirm, and help narrow.


On Jul 8, 2017 3:00 AM, "Steffen"  wrote:

> Received more details:
>
> Our client/server system needs encrypting to client.
> Almost files are statically encrypt, but some xml need to
> modify by SSI then encrypt. So I made encrypting file
> filter by exe (not Apache module). Its name is encect.exe
> (x86) and it is using msvcrt.dll. Because encect.exe works
> all Apache HTTP Server (x86/x64) which is enabled built-in
> mod_ext_filter module.
>
> (Details)
>
> Some directory uses SYMLINKDed directory. Apache HTTP
> Server too. Enabed symlink by .htaccess file in parent of
> symlink.
>
>Options +FollowSymLinks
>
> The modification & encrypt SSI template xml file are in
> the above symlinked directory.
>
> Apache HTTP Server's httpd.conf defines like follow.
>
> ExtFilterDefine ENCECT mode=output cmd=modules/encect.exe
> Alias /WebRoot/ "/Path/To/WebRoot/"
> 
>  Options Indexes MultiViews IncludesNOEXEC
>  AddOutputFilter INCLUDES m3s inc ini
>  AddOutputFilter INCLUDES;ENCECT xmlpp
>  AllowOverride All
>  Require all granted
> 
>
> The xmlpp file size is 3,295 bytes. written in Shift_JIS.
> Used SSI tag is three  like
> follow.
>
>/...
>
> Client access to xmlpp file, the encect.exe process is
> remains in service. I'm memory dump encect.exe and load to
> WinDbg. Call stack is ReadFile(), it means encect.exe
> waiting stdin, but Apache does not send any. Kill the
> encect.exe, client receives body from first SSIed parsed
> text. (Loose until first SSI tag.)
>
> I tested INCLUDE;ENCECT to..
> (A) INCLUDES only, client receives plain complete SSI
> parsed xmlpp file. OK.
> (B) ENCECT only, client receives responce header without
> body. NG. This must encrypt xmlpp file.
>
> Next, I tested mod_ext_filter.so to 2.4.25, but still NG.
>
> Next, I read Apache HTTP Server Change log, I found
> mod_filter changes. I tested mod_filter.so and
> mod_ext_filter.so to 2.4.25, NG.
>
> Next, I tested httpd.exe to 2.4.25, NG.
>
> Next, I read Apache Lounge's used modules, I found APR is
> upgraded to 1.6.2 from 1.5.x. So I tested libapr-1.dll,
> libaprutil-1.dll to 2.4.25, OK! Works like 2.4.25 and
> earlier version of Apache HTTP Server.
>
>
>
> On Friday 07/07/2017 at 20:13, William A Rowe Jr wrote:
>
> On Fri, Jul 7, 2017 at 7:13 AM, Jim Jagielski  wrote:
>
> 2.4.26/27 doesn't *require* APR/APU 1.6.x, but there are
> some features that depend on it. If it's a bug in apr 1.6.x,
> then it's not a httpd bug specifically... imo at least.
>
> any further detail on how the below is actually borken??
> What happens?
>
> On Jul 7, 2017, at 7:41 AM, Steffen  wrote:
>
>
> I got the following report. Is this known ?
>
>
> Because apache http server all of 2.4.26 (vc11,vc14,vc15)
> was not work ExtFilterDefine & OutputFilter. Its bug
> exists in apr 1.6. I thought it need to inform.
>
> Apache 2.4.26 changes apr 1.6, but it broke
> ExtFilterDefine & OutputFilter.
> (test) copy apache 2.4.25's libapr-1 & libaprutil to
> apache 2.4.26, they worked correctly like before apaches.
>
>
> Yes, we need the actual error messages.
>
> I'm about ready to T apr 1.6.1 / apu 1.6.3 for the various fixes already
> present, but if we can fix something specific that we are unaware of...
>
>
>

Re: svn commit: r1730079 - in /httpd/httpd/trunk: Makefile.in acinclude.m4 build/config_vars.sh.in

[Jacob Champion]

Some time this week, I'm going to propose this patch for backport, since 
I rely on it for the `make check` feature and I want to get that into 2.4.x.


Rainer, if you (or anyone else) have reservations, please let me know.

--Jacob

On 02/12/2016 09:46 AM, rj...@apache.org wrote:

Author: rjung
Date: Fri Feb 12 17:46:38 2016
New Revision: 1730079

URL: http://svn.apache.org/viewvc?rev=1730079=rev
Log:
Use different variables to track normal
modules and MPMs during build.

Normal modules and MPMs follow different
rules in the config, e.g. we are only
allowed to have one active LoadModule
for an MPM in the config.

As a side effect, LoadModule for MPMs
will now come before LoadModule for
the normal modules.

Modified:
 httpd/httpd/trunk/Makefile.in
 httpd/httpd/trunk/acinclude.m4
 httpd/httpd/trunk/build/config_vars.sh.in

Modified: httpd/httpd/trunk/Makefile.in
URL: 
http://svn.apache.org/viewvc/httpd/httpd/trunk/Makefile.in?rev=1730079=1730078=1730079=diff
==
--- httpd/httpd/trunk/Makefile.in (original)
+++ httpd/httpd/trunk/Makefile.in Fri Feb 12 17:46:38 2016
@@ -45,7 +45,7 @@ install-conf:
if [ -f $$i ] ; then \
( \
n_lm=`awk 'BEGIN {n=0} /@@LoadModule@@/ {n+=1} END {print 
n}' < $$i`; \
-   if test $$n_lm -eq 0 -o "x$(DSO_MODULES)" = "x"; then \
+   if test $$n_lm -eq 0 -o "x$(MPM_MODULE)$(DSO_MODULES)" = 
"x"; then \
sed -e 's#@@ServerRoot@@#$(prefix)#g' \
-e 's#@@Port@@#$(PORT)#g' \
-e 's#@@SSLPort@@#$(SSLPORT)#g' \
@@ -68,29 +68,38 @@ install-conf:
else \
have_cgid="0"; \
fi; \
+   for j in $(MPM_MODULES) "^EOL^"; do \
+   if test $$j != "^EOL^"; then \
+   if echo ",$(ENABLED_MPM_MODULE),"|$(EGREP) 
",$$j," > /dev/null ; then \
+   loading_disabled=""; \
+   else \
+   loading_disabled="#"; \
+   fi; \
+   echo "$${loading_disabled}LoadModule 
$${j}_module $(rel_libexecdir)/mod_$${j}.so"; \
+   fi; \
+   done; \
for j in $(DSO_MODULES) "^EOL^"; do \
if test $$j != "^EOL^"; then \
if echo ",$(ENABLED_DSO_MODULES),"|$(EGREP) 
",$$j," > /dev/null ; then \
loading_disabled=""; \
else \
loading_disabled="#"; \
-   mpm=`echo $$j|sed 
s/_.*//`; \
-   if test "$(LOAD_ALL_MODULES)" = "yes" -a 
"$$mpm" != "mpm"; then \
+   if test "$(LOAD_ALL_MODULES)" = 
"yes"; then \

loading_disabled=""; \
fi; \
fi; \
-   if test $$j = "cgid" -a 
"$$have_cgi" = "1"; then \
-   echo ""; \
-   echo " 
$${loading_disabled}LoadModule $${j}_module $(rel_libexecdir)/mod_$${j}.so"; \
-   echo 
""; \
-   elif test $$j = "cgi" -a 
"$$have_cgid" = "1"; then \
-   echo ""; \
-   echo " 
$${loading_disabled}LoadModule $${j}_module $(rel_libexecdir)/mod_$${j}.so"; \
-   echo 
""; \
-   else \
-   echo 
"$${loading_disabled}LoadModule $${j}_module $(rel_libexecdir)/mod_$${j}.so"; \
-   fi; \
+   if test $$j = "cgid" -a "$$have_cgi" = 
"1"; then \
+   echo ""; \
+   echo " 

Re: rfc1123 aka Proxy balancer://127

[Yann Ylavic]

Hi Jean-Frederic,

On Mon, Jul 10, 2017 at 8:56 AM, jean-frederic clere  wrote:
>
> According rfc1123 the configuration:
> ProxyPass "/" "balancer://127"
> 
> BalancerMember ajp://tomcat1:8009
> BalancerMember ajp://tomcat2:8009
> 
>
> 127 looks valid but it is rejected by httpd, apr_parse_addr_port() takes
> 127 as port instead hostname.

I think apr_parse_addr_port() is more meant to parse some special
httpd directives (which can be port only) than rfc1123's hosts.

>
> So addr is NULL, then I have:
> "AH01157: error parsing URL //127: Invalid host/port"
>
> Is 127 really valid, if yes do we plan to fix that?

We probably should use apr_uri_parse() in ap_proxy_canon_netloc(), and
possibly for performance reasons even try to do it one and only once
by request in the whole mod_proxy processing...


Regards,
Yann.

Re: mod_proxy_fcgi and flush

[Luca Toscano]

Hi Rüdiger,

2017-07-10 8:31 GMT+02:00 Plüm, Rüdiger, Vodafone Group <
ruediger.pl...@vodafone.com>:

>
>
>
>
> *Von:* Luca Toscano [mailto:toscano.l...@gmail.com]
> *Gesendet:* Samstag, 8. Juli 2017 09:52
> *An:* Apache HTTP Server Development List 
> *Betreff:* Re: mod_proxy_fcgi and flush
>
>
>
> Hi Jacob, Helmut!
>
>
>
> 2017-07-06 20:54 GMT+02:00 Jacob Champion :
>
> On 07/06/2017 11:13 AM, Jim Jagielski wrote:
>
> works 4 me...
>
>
> Doesn't for me. E.g. with a script like
>
>print("hi!\n")
>   flush();
>   sleep(1);
>   print("hi!\n");
> ?>
>
> it takes 1 second to receive a single chunk with both lines in it.
>
> From a quick skim I assume this is because we don't use nonblocking
> sockets in the proxy implementation. (There's even a note in mod_proxy_fcgi
> that says, "Yes it sucks to [get the actual data] in a second recv call,
> this will eventually change when we move to real nonblocking recv calls.")
>
>
>
> Quick check from my side too, so not sure if it makes sense. IIUC the
> comment is about getting all the headers from the FCGI backend
> (get_data_full(..., AP_FCGI_HEADER_LEN)), then get the response body in
> another one (the [actual data]).
>
>
>
> I checked mod_fcgi as Helmut suggested and it seems to me that the -flush
> feature is a simple "flush every data that you receive", so I tested the
> following patch with Jacob's php example code and it seems doing what
> Helmut asked (please correct me if I am wrong).
>
>
>
> Caveat: I had to set output_buffering = Off in my php-fpm's php.ini config
> file.
>
>
>
> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_
> fcgi-force_flush.patch
>
>
>
> Flushing unconditionally is a bad idea performance wise. Please have a
> look how ajp solved this issue:
>
>
>
> https://svn.apache.org/r327185
>
> https://svn.apache.org/r384580
>
> https://svn.apache.org/r390210
>
> https://svn.apache.org/viewvc/httpd/httpd/trunk/modules/
> proxy/mod_proxy_ajp.c?r1=325879=390210
>
>
>
> Hint: The above diff contains further unrelated changes.
>
>
>
Definitely, I added that quick and dirty patch to show my idea, really far
from being ready for a commit :)
Will review the commits that you posted, thanks!

Luca

rfc1123 aka Proxy balancer://127

[jean-frederic clere]

Hi,

According rfc1123 the configuration:
ProxyPass "/" "balancer://127"

BalancerMember ajp://tomcat1:8009
BalancerMember ajp://tomcat2:8009


127 looks valid but it is rejected by httpd, apr_parse_addr_port() takes
127 as port instead hostname.

So addr is NULL, then I have:
"AH01157: error parsing URL //127: Invalid host/port"

Is 127 really valid, if yes do we plan to fix that?

Cheers

Jean-Frederic

Re: 2.4.27

[Reindl Harald]

Am 06.07.2017 um 19:28 schrieb Jacob Champion:
Administrators using prefork who would like to switch to HTTP/2 in the 
future need to understand the limitations of the prefork architecture 
they have selected. And sure, our users can request that we implement a 
solution that "just works" with prefork, with the parent dispatcher that 
Stefan has mentioned, and we can weigh the cost/benefit of implementing 
it. But IMO it's not that onerous to ask our users to upgrade to a 
modern MPM if they want a nice new protocol


well, when i see how fragile the combination of httpd and php-fpm still 
is, thanks, but no


Apache:
COMPATIBILITY: mod_proxy_fcgi: Revert to 2.4.20 FCGI behavior for the 
default ProxyFCGIBackendType, fixing a regression with PHP-FPM. PR 61202


PHP:
Fixed bug #74738 (Multiple [PATH=] and [HOST=] sections not properly 
parsed).

https://bugs.php.net/bug.php?id=74738

PROXY protocol support

[Jim Jagielski]

Now that 2.4.27 is (almost) out, there are some other
patches in STATUS that would be good to finalize, including
in particular, the support for the HAproxy PROXY protocol
support. To help, I've gone ahead and created an updated,
merge-clean patch as well as created tests for it in the
Perl test-framework.

All we need is 1 more vote ;)

AW: rfc1123 aka Proxy balancer://127

[Plüm , Rüdiger , Vodafone Group]

> -Ursprüngliche Nachricht-
> Von: Jim Jagielski [mailto:j...@jagunet.com]
> Gesendet: Montag, 10. Juli 2017 15:33
> An: dev@httpd.apache.org
> Betreff: Re: rfc1123 aka Proxy balancer://127
> 
> Personally, I think we simply document that balancer names
> must start w/ a character.

+1

Regards

Rüdiger

> 
> > On Jul 10, 2017, at 2:56 AM, jean-frederic clere 
> wrote:
> >
> > Hi,
> >
> > According rfc1123 the configuration:
> > ProxyPass "/" "balancer://127"
> > 
> >BalancerMember ajp://tomcat1:8009
> >BalancerMember ajp://tomcat2:8009
> > 
> >
> > 127 looks valid but it is rejected by httpd, apr_parse_addr_port()
> takes
> > 127 as port instead hostname.
> >
> > So addr is NULL, then I have:
> > "AH01157: error parsing URL //127: Invalid host/port"
> >
> > Is 127 really valid, if yes do we plan to fix that?
> >
> > Cheers
> >
> > Jean-Frederic

[RESULT] [VOTE] Release httpd-2.2.34

[William A Rowe Jr]

On Thu, Jul 6, 2017 at 2:33 PM, William A Rowe Jr  wrote:
> For your consideration... pre-release candidate tarballs of
> Apache legacy httpd 2.2.34 can be found in;
>
> http://httpd.apache.org/dev/dist/
>
> Thanks all who merged the security work in and other fixes,
> and helped identify a couple more lingering defects.
>
> As we picked end of maintenance Jul 1 '17 - the [discuss]
> thread had sufficient time for response - and no other 2.4
> regressions relate to what has been ported to 2.2... it looks
> like this is the second attempt at 2.2's final hurrah...
>
> Your votes on two decisions, please?
>
> +/-1
> [ ] Release 2.2.34 as legacy GA

Passes 6 +1/no -1. Thanks for reviewing!



> [ ] Retire the 2.2.x branch from any further maintenance.

Passes 5 +1/one -0.

Yann's comment was noted, we as a development community
could look at an especially egregious security defect and decide
it was necessary to make a release. But in the interim, we can
stick with

Re: rfc1123 aka Proxy balancer://127

[Jim Jagielski]

Personally, I think we simply document that balancer names
must start w/ a character.

> On Jul 10, 2017, at 2:56 AM, jean-frederic clere  wrote:
> 
> Hi,
> 
> According rfc1123 the configuration:
> ProxyPass "/" "balancer://127"
> 
>BalancerMember ajp://tomcat1:8009
>BalancerMember ajp://tomcat2:8009
> 
> 
> 127 looks valid but it is rejected by httpd, apr_parse_addr_port() takes
> 127 as port instead hostname.
> 
> So addr is NULL, then I have:
> "AH01157: error parsing URL //127: Invalid host/port"
> 
> Is 127 really valid, if yes do we plan to fix that?
> 
> Cheers
> 
> Jean-Frederic

Re: PROXY protocol support

[Stefan Eissing]

Do I sense the sweet smell of monthly releases in the air?

> Am 10.07.2017 um 16:21 schrieb Jim Jagielski :
> 
> Now that 2.4.27 is (almost) out, there are some other
> patches in STATUS that would be good to finalize, including
> in particular, the support for the HAproxy PROXY protocol
> support. To help, I've gone ahead and created an updated,
> merge-clean patch as well as created tests for it in the
> Perl test-framework.
> 
> All we need is 1 more vote ;)

Re: 2.4.27

[Graham Leggett]

On 06 Jul 2017, at 5:15 PM, Stefan Eissing  wrote:

> This is not a bug, it is the collision of the processing models.
> 
> So, I think disabling it prevent user from shooting themselves in the foot. 
> If you are on prefork, you'd want the 6 parallel HTTP/1.1 connections, not h2.
> 
> Does this make sense?

It does, +1.

> PS. Yes, I know that one /could/ make parallel processes work in prefork by 
> placing the h2 dispatching in a parent process. If someone wants to implement 
> that, be my guest.

I've looked at this in some detail. There are a number of limitations that need 
lifting in the core, one of which is that connections need to be able to return 
EAGAIN. I have a plan for this, but I am slammed right now.

Regards,
Graham
—