Session support for mod_auth_basic [PATCH 60708]

[Emmanuel Dreyfus]

Hello

I posted this patch that brings mod_session support for mod_auth_basic
in trunk and 2.4.x
https://bz.apache.org/bugzilla/show_bug.cgi?id=60708

Any comment on it? The motivation was to reduce the latency of Webdav
operation from Windows: using just HTTP basic authentication, each
request is first done without credentials, hits a HTTP 401 and then
succeeds with credentials. 

With the help of a session, all requests after initial HTTP basic
authentication have a session cookie and can succeed on first attempt,
saving one HTTP request/response.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
m...@netbsd.org

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Eric Covener]

On Wed, Feb 8, 2017 at 3:31 PM, Jim Jagielski  wrote:
> I have. With PHP-FPM as well as that Perl script FPM. All works
> as expected.

In my env, the change doesn't result in any change to SCRIPT_NAME.  I
only test w/ the perl script and use *.phx vs *.php to see both paths.

AcceptPathInfo ON
AddType application/x-php7-fpm .php
Action application/x-php7-fpm /fpm virtual

 SetHandler "proxy:fcgi://localhost:12345"



 SetHandler "proxy:fcgi://127.0.0.1:12345"


If I hit /info.phx/path.info, SCRIPT_NAME is /info.phx and stays that way.
If I hit /info.php/path/info, SCRIPT_NAME starts /fpm and stays that
way (if using FPM, it would see REDIRECT_* and figure out our real
paths)

(added orig string to trace locally):
[Thu Feb 09 03:47:01.331270 2017] [proxy_fcgi:trace4] [pid 26390:tid
139707007817472] mod_proxy_fcgi.c(392): [client 127.0.0.1:43100]
fpm:virtual_script: Modified SCRIPT_NAME from /info.phx to: /info.phx
[Thu Feb 09 03:49:54.050047 2017] [proxy_fcgi:trace4] [pid 26390:tid
139706991032064] mod_proxy_fcgi.c(392): [client 127.0.0.1:43168]
fpm:virtual_script: Modified SCRIPT_NAME from /fpm to: /fpm

Assuming there's some alternate path that actually does change
SCRIPT_NAME by default, we a) don't have any complaint about
SCRIPT_NAME and b) have the SetEnv thing.  If we want more options,
maybe we can stash this older SCRIPT_NAME into a new variable and show
how to copy it over SCRIPT_NAME?

-- 
Eric Covener
cove...@gmail.com

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Luca Toscano]

2017-02-08 22:37 GMT+01:00 Jacob Champion :

> On 02/08/2017 01:32 PM, Jim Jagielski wrote:
>
>> I am confused then... what else are you proposing? Eric's envar fix allows
>> for people to basically adjust at their whim. What else is needed??
>>
>
> In my view, nothing else is needed. We revert my changes (and the followup
> query string modification) in 2.4.x, and backport Eric's envvar directive
> by itself to make up for it.
>
> We can sit on other SCRIPT_FILENAME changes, and the Backend directive, in
> trunk until PHP wants to sit down with us to decide what SCRIPT_FILENAME
> actually means to FPM.
>
>
FWIW my 2 cents: Jim's view and Jacob/Eric's are all valuable, but since
there is a lot of instability for mod_proxy_fcgi I'd like more to proceed
with the minimal set of changes for the next 2.4 release. We revert to a
known "good" state, we introduce the new ProxyFCGISetEnvIf directive and
listen to what our users think about it. In the meantime, we work with PHP
on Jim's solution and we think about a second backport later on if needed.
Everybody is happy :)

Luca

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jacob Champion]

On 02/08/2017 01:32 PM, Jim Jagielski wrote:

I am confused then... what else are you proposing? Eric's envar fix allows
for people to basically adjust at their whim. What else is needed??


In my view, nothing else is needed. We revert my changes (and the 
followup query string modification) in 2.4.x, and backport Eric's envvar 
directive by itself to make up for it.


We can sit on other SCRIPT_FILENAME changes, and the Backend directive, 
in trunk until PHP wants to sit down with us to decide what 
SCRIPT_FILENAME actually means to FPM.


--Jacob

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jim Jagielski]

> On Feb 8, 2017, at 4:09 PM, Jacob Champion  wrote:
> 
> On 02/08/2017 12:31 PM, Jim Jagielski wrote:
>> Have you even TRIED it?
> 
> Yes, with the latest trunk, a config of
> 
>
>SetHandler "proxy:fcgi://localhost:10102/"
>
> 
> leads to a SCRIPT_FILENAME of 
> 'proxy:fcgi://localhost:10102//tmp/apache-svn-trunk/htdocs/hello.php'.
> 
>> I have. With PHP-FPM as well as that Perl script FPM. All works
>> as expected.
> 
> I understand that it works for the use cases you have tested. My change 
> worked for the use cases I tested, and Eric's follow-up change worked for the 
> use cases he tested, but we keep finding new regressions. This is a high-risk 
> code path, and the proposed backport makes it more complex instead of less 
> IMO.
> 
> There's a good chance that the core of your addition will be part of the 
> eventual solution. My point is that without fixing the other stuff at the 
> same time, we've just added yet another type of almost-correct behavior that 
> we and the FCGI implementations have to code around.
> 

I am confused then... what else are you proposing? Eric's envar fix allows
for people to basically adjust at their whim. What else is needed??

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jacob Champion]

On 02/08/2017 12:31 PM, Jim Jagielski wrote:

Have you even TRIED it?


Yes, with the latest trunk, a config of


SetHandler "proxy:fcgi://localhost:10102/"


leads to a SCRIPT_FILENAME of 
'proxy:fcgi://localhost:10102//tmp/apache-svn-trunk/htdocs/hello.php'.



I have. With PHP-FPM as well as that Perl script FPM. All works
as expected.


I understand that it works for the use cases you have tested. My change 
worked for the use cases I tested, and Eric's follow-up change worked 
for the use cases he tested, but we keep finding new regressions. This 
is a high-risk code path, and the proposed backport makes it more 
complex instead of less IMO.


There's a good chance that the core of your addition will be part of the 
eventual solution. My point is that without fixing the other stuff at 
the same time, we've just added yet another type of almost-correct 
behavior that we and the FCGI implementations have to code around.


--Jacob

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jim Jagielski]

Have you even TRIED it?

I have. With PHP-FPM as well as that Perl script FPM. All works
as expected.

> On Feb 8, 2017, at 3:27 PM, Jacob Champion  wrote:
> 
> On 02/08/2017 12:10 PM, Jim Jagielski wrote:
>> Doesn't the below make it work without changes.
>> 
>> #define FCGI_MAY_BE_FPM(dconf)  \
>>(dconf &&   \
>>((dconf->backend_type == BACKEND_DEFAULT_UNKNOWN) || \
>>(dconf->backend_type == BACKEND_FPM)))
> 
> No. (To "make it work" we have to cover a gigantic number of use cases, not 
> just PHP-FPM, which is probably why we're talking past each other.)
> 
> With your proposed backport, any users who were able to start using FastCGI 
> backends that required an "actual" SCRIPT_FILENAME, in the 2.4.21 or later 
> timeframe, will still have to modify their configs to specify a generic 
> FastCGI backend so that the "proxy:fcgi" stripping can kick in. Existing 
> PHP-FPM users don't change their config, sure, but they'll see yet another 
> behavior change because of the new fixup, which is just too risky IMO.
> 
> Compare that to a complete reversion to the previous behavior, plus the 
> addition of the new FCGI SetEnv directive. The users requiring an "actual" 
> SCRIPT_FILENAME will be able to set it manually, which is no worse than 
> having to set the new Backend directive (better, actually, because 
> SCRIPT_FILENAME has no globally accepted meaning). Existing PHP-FPM users are 
> guaranteed the *exact* behavior they saw before 2.4.21, making it less risky 
> to upgrade.
> 
> --Jacob

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jacob Champion]

On 02/08/2017 12:10 PM, Jim Jagielski wrote:

Doesn't the below make it work without changes.

#define FCGI_MAY_BE_FPM(dconf)  \
(dconf &&   \
((dconf->backend_type == BACKEND_DEFAULT_UNKNOWN) || \
(dconf->backend_type == BACKEND_FPM)))


No. (To "make it work" we have to cover a gigantic number of use cases, 
not just PHP-FPM, which is probably why we're talking past each other.)


With your proposed backport, any users who were able to start using 
FastCGI backends that required an "actual" SCRIPT_FILENAME, in the 
2.4.21 or later timeframe, will still have to modify their configs to 
specify a generic FastCGI backend so that the "proxy:fcgi" stripping can 
kick in. Existing PHP-FPM users don't change their config, sure, but 
they'll see yet another behavior change because of the new fixup, which 
is just too risky IMO.


Compare that to a complete reversion to the previous behavior, plus the 
addition of the new FCGI SetEnv directive. The users requiring an 
"actual" SCRIPT_FILENAME will be able to set it manually, which is no 
worse than having to set the new Backend directive (better, actually, 
because SCRIPT_FILENAME has no globally accepted meaning). Existing 
PHP-FPM users are guaranteed the *exact* behavior they saw before 
2.4.21, making it less risky to upgrade.


--Jacob

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jim Jagielski]

> On Feb 8, 2017, at 2:49 PM, Jacob Champion  wrote:
> 
> On 02/08/2017 11:32 AM, Jim Jagielski wrote:
>> It does it automatically requiring no config-file changes
>> to the end user.
> 
> Anyone picking up the SCRIPT_FILENAME change (which was my change that 
> started this whole mess) still has to change their backend type manually, so 
> I'm not convinced.
> 

??

Doesn't the below make it work without changes.

#define FCGI_MAY_BE_FPM(dconf)  \
(dconf &&   \
((dconf->backend_type == BACKEND_DEFAULT_UNKNOWN) || \
(dconf->backend_type == BACKEND_FPM)))


Unless you mean that those who "fixed" their configs due to 2.4.24/25
will need to do so again?

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jacob Champion]

On 02/08/2017 11:32 AM, Jim Jagielski wrote:

It does it automatically requiring no config-file changes
to the end user.


Anyone picking up the SCRIPT_FILENAME change (which was my change that 
started this whole mess) still has to change their backend type 
manually, so I'm not convinced.


But reverting to previous behavior *is* an automatic fix, since we know 
PHP-FPM already performs its fixups. Then people can manually edit 
SCRIPT_FILENAME if needed until we get our act together alongside PHP. 
We don't even have to ship the Backend directive until we're sure that's 
the direction we want to go.


--Jacob

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jim Jagielski]

> On Feb 8, 2017, at 1:08 PM, Jacob Champion  wrote:
> 
> On 02/08/2017 07:52 AM, Eric Covener wrote:
>> My concern is that nobody ever reported this config as broken and
>> there's no telling what any on-by-default change here can break.
> 
> +1.
> 
> Maybe a different way to put it: what does this approach solve that a 
> revert-to-previous-behavior + FCGI-SetEnv doesn't solve already?
> 

It does it automatically requiring no config-file changes
to the end user.

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jacob Champion]

On 02/08/2017 07:52 AM, Eric Covener wrote:

My concern is that nobody ever reported this config as broken and
there's no telling what any on-by-default change here can break.


+1.

Maybe a different way to put it: what does this approach solve that a 
revert-to-previous-behavior + FCGI-SetEnv doesn't solve already?


--Jacob

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jim Jagielski]

Just to be clear, I simply *tested* against that config... the patch
is not designed to do anything with it. It was basically to ensure
that it introduced no regressions.

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jim Jagielski]

> On Feb 8, 2017, at 10:52 AM, Eric Covener  wrote:
> 
> On Wed, Feb 8, 2017 at 10:45 AM, Jim Jagielski  wrote:
>> Which ones? You mean the special case one?
>> 
>> Would it be better if we did that *before* calling
>> the fix_cgivars(). What is there makes the following
>> config pair work as-is:
>> 
>>  AddType application/x-php7-fpm .php
>>  Action application/x-php7-fpm /fpm virtual
>>  
>>   SetHandler proxy:fcgi://localhost:9001
>>  
>> 
>>--
>> 
>>  
>>SetHandler "proxy:fcgi://localhost:9001
>>  
> 
> 
> My concern is that nobody ever reported this config as broken and
> there's no telling what any on-by-default change here can break.

But this is covered isn't it, but if one uses ProxyFCGIBackendType.
It's a noop if desired for that particular use-case.

How about if we move this segment to above fix_cgivars() and
give a head's up to PHP that 2.4-HEAD includes this...

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Eric Covener]

On Wed, Feb 8, 2017 at 10:45 AM, Jim Jagielski  wrote:
> Which ones? You mean the special case one?
>
> Would it be better if we did that *before* calling
> the fix_cgivars(). What is there makes the following
> config pair work as-is:
>
>   AddType application/x-php7-fpm .php
>   Action application/x-php7-fpm /fpm virtual
>   
>SetHandler proxy:fcgi://localhost:9001
>   
>
> --
>
>   
> SetHandler "proxy:fcgi://localhost:9001
>   


My concern is that nobody ever reported this config as broken and
there's no telling what any on-by-default change here can break.

Re: svn commit: r1782209 - /httpd/httpd/branches/2.4.x/STATUS

[Jim Jagielski]

Which ones? You mean the special case one?

Would it be better if we did that *before* calling
the fix_cgivars(). What is there makes the following
config pair work as-is:

  AddType application/x-php7-fpm .php
  Action application/x-php7-fpm /fpm virtual
  
   SetHandler proxy:fcgi://localhost:9001
  

--

  
SetHandler "proxy:fcgi://localhost:9001
  

By allowing fix_cgivars() after that, then we could work around
those rare cases when even that fixup isn't correct for the users
environ.

> On Feb 8, 2017, at 10:28 AM, cove...@apache.org wrote:
> 
> Author: covener
> Date: Wed Feb  8 15:28:56 2017
> New Revision: 1782209
> 
> URL: http://svn.apache.org/viewvc?rev=1782209=rev
> Log:
> danger?
> 
> Modified:
>httpd/httpd/branches/2.4.x/STATUS
> 
> Modified: httpd/httpd/branches/2.4.x/STATUS
> URL: 
> http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1782209=1782208=1782209=diff
> ==
> --- httpd/httpd/branches/2.4.x/STATUS (original)
> +++ httpd/httpd/branches/2.4.x/STATUS Wed Feb  8 15:28:56 2017
> @@ -223,6 +223,7 @@ PATCHES PROPOSED TO BACKPORT FROM TRUNK:
>   http://svn.apache.org/r1782194
>  2.4.x patch: http://home.apache.org/~jim/patches/mod_proxy_fcgi.patch
>  +1: jim
> + covener: -0.9: new fixups in here seem risky, do we really want these?  
> 
> 
> PATCHES/ISSUES THAT ARE BEING WORKED
> 
> 

FINAL REMINDER: CFP for ApacheCon closes February 11th

[Rich Bowen]

Dear Apache Enthusiast,

This is your FINAL reminder that the Call for Papers (CFP) for ApacheCon
Miami is closing this weekend - February 11th. This is your final
opportunity to submit a talk for consideration at this event.

This year, we are running several mini conferences in conjunction with
the main event, so if you're submitting for one of those events, please
pay attention to the instructions below.

Apache: Big Data
* Event information:
http://events.linuxfoundation.org/events/apache-big-data-north-america
* CFP:
http://events.linuxfoundation.org/events/apache-big-data-north-america/program/cfp

Apache: IoT (Internet of Things)
* Event Information: http://us.apacheiot.org/
* CFP -
http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp
(Indicate 'IoT' in the Target Audience field)

CloudStack Collaboration Conference
* Event information: http://us.cloudstackcollab.org/
* CFP -
http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp
(Indicate 'CloudStack' in the Target Audience field)

FlexJS Summit
* Event information - http://us.apacheflexjs.org/
* CFP -
http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp
(Indicate 'Flex' in the Target Audience field)

TomcatCon
* Event information - https://tomcat.apache.org/conference.html
* CFP -
http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp
(Indicate 'Tomcat' in the Target Audience field)

All other topics and projects
* Event information -
http://events.linuxfoundation.org/events/apachecon-north-america/program/about
* CFP -
http://events.linuxfoundation.org/events/apachecon-north-america/program/cfp

Admission to any of these events also grants you access to all of the
others.

Thanks, and we look forward to seeing you in Miami!

-- 
Rich Bowen
VP Conferences, Apache Software Foundation
rbo...@apache.org
Twitter: @apachecon



(You are receiving this email because you are subscribed to a dev@ or
users@ list of some Apache Software Foundation project. If you do not
wish to receive email from these lists any more, you must follow that
list's unsubscription procedure. View the headers of this message for
unsubscription instructions.)

Re: proxy test failures between 2.4 and trunk

[Jim Jagielski]

I am back w/ cycles so ping me if I can help.

> On Feb 8, 2017, at 8:47 AM, Eric Covener  wrote:
> 
> I owe a backport in this one, jorton added the test to demonstrate an
> issue reported.  Will move it along today.
> 
> On Wed, Feb 8, 2017 at 8:43 AM, Jim Jagielski  wrote:
>> Just a FYI that there is a test failure in t/modules/proxy.t
>> in 2.4-HEAD which is not present in trunk...
>> 
>> # testing : reverse Location-proxy to index.html
>> # expected: 200
>> # received: '404'
>> not ok 3
>> # testing : reverse Location-proxied body
>> # expected: qr/^welcome to /
>> # received: '
>> # 
>> # 404 Not Found
>> # 
>> # Not Found
>> # The requested URL /locproxy/ was not found on this server.
>> # 
>> # '
>> not ok 4
> 
> 
> 
> -- 
> Eric Covener
> cove...@gmail.com

Re: proxy test failures between 2.4 and trunk

[Eric Covener]

I owe a backport in this one, jorton added the test to demonstrate an
issue reported.  Will move it along today.

On Wed, Feb 8, 2017 at 8:43 AM, Jim Jagielski  wrote:
> Just a FYI that there is a test failure in t/modules/proxy.t
> in 2.4-HEAD which is not present in trunk...
>
> # testing : reverse Location-proxy to index.html
> # expected: 200
> # received: '404'
> not ok 3
> # testing : reverse Location-proxied body
> # expected: qr/^welcome to /
> # received: '
> # 
> # 404 Not Found
> # 
> # Not Found
> # The requested URL /locproxy/ was not found on this server.
> # 
> # '
> not ok 4



-- 
Eric Covener
cove...@gmail.com

proxy test failures between 2.4 and trunk

[Jim Jagielski]

Just a FYI that there is a test failure in t/modules/proxy.t
in 2.4-HEAD which is not present in trunk...

# testing : reverse Location-proxy to index.html
# expected: 200
# received: '404'
not ok 3
# testing : reverse Location-proxied body
# expected: qr/^welcome to /
# received: '
# 
# 404 Not Found
# 
# Not Found
# The requested URL /locproxy/ was not found on this server.
# 
# '
not ok 4

Re: mood_remoteip ProxyProtocol addition

[Sander Hoentjen]

On 02/08/2017 01:00 AM, Reindl Harald wrote:
>
>
> Am 08.02.2017 um 00:44 schrieb Yann Ylavic:
>> On Wed, Feb 8, 2017 at 12:25 AM, Yann Ylavic 
>> wrote:
>>> On Wed, Feb 8, 2017 at 12:01 AM, Reindl Harald
>>>  wrote:

 how can you trust as a php application developer that
 "X-Forwarded-Proto" is
 trustable and not from the enduser client at all - for REMOTE_ADDR
 you don't
 consider "X-Forwarded-For" exactly for that reason
>>>
>>> I'm not proposing to use or trust "X-Forwarded-Proto" directly, but
>>> that mod_remoteip [either directly or provides the (optional)
>>> functions for ap_add_{common,cgi}_vars() to] set REMOTE_HTTPS=on
>>> and/or REMOTE_SCHEME=https accordingly.
>>> Just like REMOTE_ADDR.
>>>
>>> But not change HTTPS and/or REQUEST_SCHEME (but more importantly their
>>> sources/hooks as accessed and read by core/modules), like (IIUC)
>>> proposed by the patches.
>>> These are local informations.
>>
>> Actually, I'm not really opposed to set HTTPS=on (according to
>> mod_remoteip) in the environment *given to the script/CGI* only, if
>> that's the trigger for it to do the desired thing, this won't be used
>> by httpd internally at least.
>>
>> What's proposed so far is much more than that (if I read the patches
>> correctly)
>
> ok, so finally we agree :-)
>
> i am only interested in a centralized way to get rid of hacks like
> below in each and every application where mod_remoteip solves the
> similar problem with $_SERVER['REMOTAE_ADDR'] for cgi/mod_php
>
> $_SERVER['REQUEST_SCHEME'] because you typically build a full
> qualifiied URL for a link in emails with $_SERVER['REQUEST_SCHEME'] .
> '//' . $_SERVER['SERVER_NAME'] . $_SERVER['REQUEST_URI'] . '?param=x'
>
> in my own application the hack below was simple - in case of other
> software like Magento and so on you have to hack "index.php" while at
> the same time you should not touch the application code to keep it
> easily updateable
>
> if(!empty($config['cms_tls_offload']))
> {
>  $_SERVER['SERVER_PORT']= '443';
>  $_SERVER['REQUEST_SCHEME'] = 'https';
>  $_SERVER['HTTPS']  = 'on';
> }
>
As the OP, first let me say sorry for the typo in the subject.
Now, I must say that I agree too about anything that has been said in
this thread so far.
There should be a standard way for applications to check information.
Unfortunately, historically the way applications(1) deal with it varies
a bit. In a .htaccess the things most seen are:

RewriteCond %{HTTPS} !=on
RewriteCond %{SERVER_PORT} ^443$ [OR]
Header set Strict-Transport-Security "max-age=31536000" env=HTTPS

Basically, if those stop working I am screwed.

Then, on to PHP:
I think most of them check the HTTPS env var ($_SERVER['HTTPS']) but
some check REQUEST_SCHEME or SERVER_PORT. Probably some users are even
more creative.

Of course, there are also those who check things like
HTTP_X_FORWARDED_PROTO in addition.

Because we have lots of customers, it would be virtually impossible to
"fix" all their applications.

Onto the subject of security:
As the system administrator of those servers, I could choose to do SSL
ofloading, non-encrypted to a proxy, then that proxy uses an Apache SSL
backend. In this case the connection is as (in)secure as the case where
the SSL-offloader talks directly to Apache. But, applications would
continue to work. On the matter of PHP applications I would like to add
that PHP-FPM also does not have encryption. From an end-user security
POV there would be (IMHO) no difference between SSL-offloader -> Apache
-> PHP-FPM and Apache -> PHP-FPM.

The thing I am trying to say here is that for the most part, it is up to
the system administrator to configure things securely. It would help if
Apache allows the sysadmin to choose. Of course it is important to think
really hard about what is allowed and what not, so I am glad this
discussion is taking place.


(1) I work at a hosting provider, so I am talking mostly about PHP
"applications"

-- 
Sander