Re: 2.4.27
On 2017-07-18 14:25, Eric Covener wrote:
> Argh, not right, missed the other return stmt.
>
> It seems like proxy_trans will return OK to translate_name() and not
> let mod_rewrite in non-perdir run at all. It is rigged to run before
> mod_rewrite.
Ok, it seems that my rewrite issue has gained a bit of attention, thus I
will explain it now in more detail (otherwise people don't know what I
am actually talking about):
Options FollowSymLinks
AllowOverride None
ErrorDocument 404 /404.php
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^([^\.]+)$ $1.php [NC,L]
Require all granted
With mod_php I got the 404.php error page when going to:
https://example.com/te
With mod_proxy_fcgi I get a "File not found." error in the browser and
the following is in Apache's error_log:
[proxy_fcgi:error] AH01071: Got error 'Primary script unknown\n'
However, I get the 404.php error page, when going to:
https://example.com/te.t
https://example.com/te/
(So it works partly with proxy_fcgi.)
I'm still investigating why that is. I'm a bit pressed for time that's
why I haven't been able to look into it in more detail.
I have to setup a VM to debug this further, but IMO this is not ok.
In both cases the rewrite should work the same way.
My handler cfg looks like this:
SetHandler "proxy:unix:/var/run/php7-fpm/sample.sock|fcgi://sample"
A rather simple use case, yet the bahaviour is different. Don't get me
wrong, but it shouldn't behave differently.
--
regards Helmut K. C. Tessarek KeyID 0xF7832007C11F128D
Key fingerprint = 28A3 1666 4FE8 D72C CFD5 8B23 F783 2007 C11F 128D
/*
Thou shalt not follow the NULL pointer for chaos and madness
await thee at its end.
*/
Re: 2.4.27
On Tue, Jul 18, 2017 at 2:24 PM, Eric Covener wrote: > On Tue, Jul 18, 2017 at 12:54 PM, Yann Ylavic wrote: >> On Tue, Jul 18, 2017 at 6:27 PM, Helmut K. C. Tessarek >> wrote: >>> >>> Have you seen this sentence? >>> > So ProxyPass has precedence over other directives. It is evaluated > first. This can lead to a number of problems. >> >> ProxyPass has no particular precedence over other directives, it's >> just that some of them have no meaning is proxy context: e.g. >> (as you mentioned). >> > > In the case of ProxyPass vs. non per-dir rewrites, you will see at > least a change REQUEST_FILENAME (which in non-perdir w/o mod_proxy > matching is the same as REQUEST_URI and really is a bad idea to use!) Argh, not right, missed the other return stmt. It seems like proxy_trans will return OK to translate_name() and not let mod_rewrite in non-perdir run at all. It is rigged to run before mod_rewrite. -- Eric Covener cove...@gmail.com
Re: 2.4.27
On Tue, Jul 18, 2017 at 12:54 PM, Yann Ylavic wrote: > On Tue, Jul 18, 2017 at 6:27 PM, Helmut K. C. Tessarek > wrote: >> >> Have you seen this sentence? >> So ProxyPass has precedence over other directives. It is evaluated first. This can lead to a number of problems. > > ProxyPass has no particular precedence over other directives, it's > just that some of them have no meaning is proxy context: e.g. > (as you mentioned). > In the case of ProxyPass vs. non per-dir rewrites, you will see at least a change REQUEST_FILENAME (which in non-perdir w/o mod_proxy matching is the same as REQUEST_URI and really is a bad idea to use!)
Re: [Bug 60956] Event MPM listener thread may get blocked by SSL shutdowns
2017-07-17 9:33 GMT+02:00 Stefan Eissing : > > > Am 14.07.2017 um 21:52 schrieb Yann Ylavic : > > > > On Fri, Jun 30, 2017 at 1:33 PM, Yann Ylavic > wrote: > >> On Fri, Jun 30, 2017 at 1:20 PM, Ruediger Pluem > wrote: > >>> > >>> On 06/30/2017 12:18 PM, Yann Ylavic wrote: > > IMHO mod_ssl shoudn't (BIO_)flush unconditionally in > modssl_smart_shutdown(), only in the "abortive" mode of > ssl_filter_io_shutdown(). > >>> > >>> I think the issue starts before that. > >>> ap_prep_lingering_close calls the pre_close_connection hook and > modules that are registered > >>> to this hook can perform all sort of long lasting blocking operations > there. > >>> While it can be argued that this would be a bug in the module I think > the only safe way > >>> is to have the whole start_lingering_close_nonblocking being executed > by a worker thread. > >> > >> Correct, that'd be much simpler/safer indeed. > >> We need a new SHUTDOWN state then, right? > > > > Actually it was less simple than expected, and it has some caveats > obviously... > > > > The attached patch does not introduce a new state but reuses the > > existing CONN_STATE_LINGER since it was not really considered by the > > listener thread (which uses CONN_STATE_LINGER_NORMAL and > > CONN_STATE_LINGER_SHORT instead), but that's a detail. > > > > Mainly, start_lingering_close_nonblocking() now simply schedules a > > shutdown (i.e. pre_close_connection() followed by immediate close) > > that will we be run by a worker thread. > > A new shutdown_linger_q is created/handled (with the same timeout as > > the short_linger_q, namely 2 seconds) to hold connections to be > > shutdown. > > > > So now when a connection times out in the write_completion or > > keepalive queues, it needs (i.e. the listener may wait for) an > > available worker to process its shutdown/close. > > This means we can *not* close kept alive connections immediatly like > > before when becoming short of workers, which will favor active KA > > connections over new ones in this case (I don't think it's that > > serious but the previous was taking care of that. For me it's up to > > the admin to size the workers appropriately...). > > > > Same when a connection in the shutdown_linger_q itself times out, the > > patch will require a worker immediatly to do the job (see > > shutdown_lingering_close() callback). > > > > So overall, this patch may introduce the need for more workers than > > before, what was (wrongly) done by the listener thread has to be done > > somewhere anyway... > > > > Finally, I think there is room for improvements like batching > > shutdowns in the same worker if there is no objection on the approach > > so far. > > > > WDYT? > > I will test the patch, most likely today. I lot of +1s for the initiative! > > Thanks a lot for this work Yann, really nice! I will try to test it as well during the next days, I was not convinced in the beginning that triggering a (potential) increase in workers usage was super great for our users but it is definitely the most correct and safe thing to do. Even if we find a way to fix the ssl-lingering-close-block issue we might encounter a similar one in the future, so it is better imho to fix it at the source. Luca
Re: 2.4.27
On 2017-07-18 12:54, Yann Ylavic wrote: > You should probably re-check with latest 2.4.27, where some > regressions with regard to php-fpm (since 2.4.20) where finally > addressed (see https://bz.apache.org/bugzilla/show_bug.cgi?id=61202). I did. 2.4.27 fixed a proxy_fcgi problem and nextcloud: https://github.com/nextcloud/server/issues/5660 It didn't help with the rewrite issue. -- regards Helmut K. C. Tessarek KeyID 0xF7832007C11F128D Key fingerprint = 28A3 1666 4FE8 D72C CFD5 8B23 F783 2007 C11F 128D /* Thou shalt not follow the NULL pointer for chaos and madness await thee at its end. */
Re: 2.4.27
On Tue, Jul 18, 2017 at 6:27 PM, Helmut K. C. Tessarek wrote: > > Have you seen this sentence? > >>> So ProxyPass has precedence over other directives. It is evaluated >>> first. This can lead to a number of problems. ProxyPass has no particular precedence over other directives, it's just that some of them have no meaning is proxy context: e.g. (as you mentioned). and apply to local/served paths and files (served), while proxying is about remote/forwarded ones, that is: a . > > I stated that mod_rewrite behaves differently when using mod_php and > mod_proxy_fcgi. This sounds more like a bug, because it really shouldn't. > > I used rewrite with mod_php. It worked. > Then I switched to mod_proxy_fcgi. Now it does not. > > So why is that? I also mentioned I have to debug it further. You should probably re-check with latest 2.4.27, where some regressions with regard to php-fpm (since 2.4.20) where finally addressed (see https://bz.apache.org/bugzilla/show_bug.cgi?id=61202). Regards, Yann.
Re: 2.4.27
Hi David, Thanks for your reply, but I have already established in my previous email what the order of evaluation is. Have you seen this sentence? >> So ProxyPass has precedence over other directives. It is evaluated >> first. This can lead to a number of problems. On 2017-07-18 09:33, David Zuelke wrote: > You need to use SetHandler. You can't use rewrites with ProxyPass > because of the order of evaluation. I am. I have never used anything else but SetHandler. I think I explained in the mail you replied to what I think of ProxyPass. > Further questions should be taken to the users list though, as this > one here is for development of httpd... I believe you did not read my email or read something into it. This is not a user question. I know how to use mod_rewrite. I stated that mod_rewrite behaves differently when using mod_php and mod_proxy_fcgi. This sounds more like a bug, because it really shouldn't. I used rewrite with mod_php. It worked. Then I switched to mod_proxy_fcgi. Now it does not. So why is that? I also mentioned I have to debug it further. -- regards Helmut K. C. Tessarek KeyID 0xF7832007C11F128D Key fingerprint = 28A3 1666 4FE8 D72C CFD5 8B23 F783 2007 C11F 128D /* Thou shalt not follow the NULL pointer for chaos and madness await thee at its end. */
Re: 2.4.27
You need to use SetHandler. You can't use rewrites with ProxyPass because of the order of evaluation. Example config: Define php-fpm unix:/tmp/php-fpm.sock|fcgi://php-fpm # make sure the proxy is registered with the unix socket; we can then use just "fcgi://php-fpm" in proxy and rewrites directives # we have to do this because we can't rewrite to a UDS location and because PHP doesn't support the unix:...|fcgi://... syntax # this is also a lot more convenient for users # http://thread.gmane.org/gmane.comp.apache.devel/52892 # we must declare a parameter in here or it'll not register the proxy ahead of time # min=0 is an obvious candidate since that's the default value already and sensible ProxySet min=0 # pass requests to .php files to mod_proxy_fcgi # this requires Apache 2.4.10+ and PHP 5.5.15+ to work properly # make sure the file exists so that if not, Apache will show its 404 page and not FPM SetHandler proxy:fcgi://php-fpm You can then simply rewrite to index.php etc, and it'll work. Further questions should be taken to the users list though, as this one here is for development of httpd... > On 17. Jul 2017, at 19:25, Helmut K. C. Tessarek wrote: > > On 2017-07-17 03:50, Luca Toscano wrote: >> mod-proxy-fcgi is the preferred solution over mod-fcgi, and we have >> documentation about it. Any specific reason to use libapache2-mod-fcgid? >> (asking for curiosity and/or to decide if a doc update is needed :) > > I am using mod_proxy_fcgi exactly for that reason (stated on > https://wiki.apache.org/httpd/php). But the documentation > (https://wiki.apache.org/httpd/PHP-FPM) is IMO a bit off. > >> Can you please be more specific? What errors do you see? In case please >> open a task in bugzilla so we'll be able to debug it :) > > Even according to the documentation for mod_proxy_fcgi, UDS does not > support connection reuse. > In my case it broke POST requests. I then googled and found a bunch of > articles and stackexchange entries that suggested to remove the > enablereuse=on option from the Proxy section. > > Only after removing the Proxy directive completely, everything started > to work as expected. > > Except from the mod_rewrite issue I experience. I'm still debugging it, > but mod_rewrite behaves differently between mod_php and mod_proxy_fcgi, > which should not happen at all, since rewrite shouldn't care or know > about the backend. I also googled and found a few entries, none of which > helped me: > > https://stackoverflow.com/questions/44054617/mod-rewrite-in-2-4-25-triggering-fcgi-primary-script-unknown-error-in-php-fpm > > http://www.coders.pro/2017/01/qq-htm/ > >>Using ProxyPassMatch is not only dangerous, it also has precedence over >>a FilesMatch directive, which in turn limits your ability to restrict >>access to certain resources. At least that was the case a couple of >>years back. >> >> Dangerous in what way? Can you please be more specific and/or add examples? > > I'm sorry, my bad. I should not have generalized it. ProxyPass and > ProxyPassMatch _can_ be dangerous. I see 2 main issues: > > 1) The match part can be set too wide, which could allow php-fpm to > interpret the wrong file. > > 2) The documentation also states: > Warning: when you ProxyPass a request to another server (in this case, > the php-fpm daemon), authentication restrictions, and other > configurations placed in a Directory block or .htaccess file, may be > bypassed. > > So ProxyPass has precedence over other directives. It is evaluated > first. This can lead to a number of problems. > Anyway, as long as you are aware of it, the impact can be minimized. Yet > I believe it is too dangerous for the average Joe. > > -- > regards Helmut K. C. Tessarek KeyID 0xF7832007C11F128D > Key fingerprint = 28A3 1666 4FE8 D72C CFD5 8B23 F783 2007 C11F 128D > > /* > Thou shalt not follow the NULL pointer for chaos and madness > await thee at its end. > */
Re: mod_proxy_fcgi and flush
Hello Jacob! 2017-07-18 1:25 GMT+02:00 Jacob Champion : > On 07/14/2017 02:29 AM, Luca Toscano wrote: > >> http://home.apache.org/~elukey/httpd-2.4.x-mod_proxy_fcgi- >> force_flush-v3.patch created, I tested it quickly and it seems working >> fine (still unsure about using r->connection->pool vs conn->pool in >> palloc). More tests following in the weekend :) >> > Some thoughts: > Thanks a lot for reviewing :) > - This will flush each time a POLLIN event is successfully handled, even > if the event doesn't cause data to be put on the brigade (as is the case > for STDERR or END_REQUEST records, or for the empty end-of-stream record). > This is true, I thought that the "if (rv != APR_SUCCESS)" right before the new code was enough to remove corner cases but I was wrong. Maybe it is only a matter of checking that clen is > 0? Will try to check it today. > - While we're waiting for a POLLIN event in auto-flush mode, we're not > listening to POLLOUT events. Seems like we could/should merge the apr_poll > calls into one. I'm playing around with that now. I didn't do it on purpose because I didn't want to mess with the main apr_poll, but if you find an elegant solution I am all for it. My understanding is that in this case we don't need to listen to POLLOUT events since we are focusing on flushing (giving priority to it), and then we'll pay attention to POLLOUT when the main apr_poll will happen. Luca
