Re: svn commit: r1752099 - in /httpd/httpd/trunk: ./ build/rpm/ docs/log-message-tags/ docs/manual/ docs/manual/mod/ modules/filters/

[Graham Leggett]

On 11 Jul 2016, at 4:39 PM, Ruediger Pluem  wrote:

>> Added: httpd/httpd/trunk/modules/filters/mod_crypto.c
>> URL: 
>> http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/mod_crypto.c?rev=1752099=auto
>> ==
>> --- httpd/httpd/trunk/modules/filters/mod_crypto.c (added)
>> +++ httpd/httpd/trunk/modules/filters/mod_crypto.c Sun Jul 10 17:27:03 2016
> 
>> +static void *merge_crypto_config(apr_pool_t * p, void *basev, void *addv)
>> +{
>> +crypto_conf *new = (crypto_conf *) apr_pcalloc(p, sizeof(crypto_conf));
>> +crypto_conf *add = (crypto_conf *) addv;
>> +crypto_conf *base = (crypto_conf *) basev;
>> +
>> +new->library = (add->library_set == 0) ? base->library : add->library;
>> +new->params = (add->library_set == 0) ? base->params : add->params;
>> +new->library_set = add->library_set || base->library_set;
>> +
>> +new->crypto = base->crypto;
> 
> Shouldn't this be:
> 
> new->crypto = add->crypto;

In this case no, the value of crypto is set globally and needs to be unique 
across the server.

>> +
>> +return (void *) new;
>> +}
>> +
>> +static void *create_crypto_dir_config(apr_pool_t * p, char *dummy)
>> +{
>> +crypto_dir_conf *new =
>> +(crypto_dir_conf *) apr_pcalloc(p, sizeof(crypto_dir_conf));
>> +
>> +new->size_set = 0;  /* unset */
> 
> Is this needed? We do apr_pcalloc above.

We don’t need it, this is fixed.

>> +new->size = DEFAULT_BUFFER_SIZE;/* default size */
>> +new->cipher = DEFAULT_CIPHER;
>> +new->cipher = DEFAULT_MODE;
> 
> Shouldn't this be:
> 
> new->mode = DEFAULT_MODE;

Definitely.

All fixed in r1752348, thanks for this.

Regards,
Graham
—

Some Windows CMake fixes in trunk

[Jacob Champion]

For those of you using the CMakeLists on Windows, there are now a few 
bugfixes in trunk. If you have any spare time to make sure that they 
function properly in your workflow, that'd be much appreciated.


Thanks!
--Jacob

ApacheCon Europe Call For Papers Open

[Rich Bowen]

As you are no doubt already aware, we will be holding ApacheCon in
Seville, Spain, the week of November 14th, 2016. The call for papers
(CFP) for this event is now open, and will remain open until 
September 9th.

The event is divided into two parts, each with its own CFP. The first
part of the event, called Apache Big Data, focuses on Big Data
projects and related technologies.

Website: http://events.linuxfoundation.org/events/apache-big-data-europe
CFP: http://events.linuxfoundation.org/events/apache-big-data-europe/program/cfp

The second part, called ApacheCon Europe, focuses on the Apache
Software Foundation as a whole, covering all projects, community
issues, governance, and so on.

Website: http://events.linuxfoundation.org/events/apachecon-europe
CFP: http://events.linuxfoundation.org/events/apachecon-europe/program/cfp

ApacheCon is the official conference of the Apache Software
Foundation, and is the best place to meet members of your project and
other ASF projects, and strengthen your project's community.

If your organization is interested in sponsoring ApacheCon, contact me
at e...@apache.org  ApacheCon is a great place to find the brightest
developers in the world, and experts on a huge range of technologies.

I hope to see you in Seville!

Re: concurrent ssl context access

[Stefan Eissing]

> Am 12.07.2016 um 15:28 schrieb William A Rowe Jr :
> 
> On Tue, Jul 12, 2016 at 3:37 AM, Stefan Eissing 
>  wrote:
> 
> > Am 11.07.2016 um 17:38 schrieb William A Rowe Jr :
> >
> > On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing 
> >  wrote:
> > In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept up 
> > that StdEnvVars makes concurrent access to the SSL context when HTTP/2 is 
> > in place.
> >
> > The question is: how do we want to fix this?
> >
> > The general theme is that a module needs to perform some action on a 
> > connection, sees that it is a slave connection and accesses the master 
> > instead. Such a module needs to be aware that access to master may happen 
> > in parallel with other request.
> >
> > Should we let each such module care for it on its own or do we want to 
> > provide a way to access the master connection in a mutex protected way?
> >
> > Rather than a mutex, which would become an issue because all master access
> > to the context would require blocking, is there a way we can allow a module 
> > to
> > perform its (hopefully swift) processing on the master thread? Send a 
> > metadata
> > bucket to the master asking for a function to be invoked?
> 
> Hmm, interesting idea. Of course, there would still be a mutex involved in 
> handling such a query, but maybe it can be done more elegant than mutexing 
> all accesses to structures everywhere.
> 
> One way to look at this is that information retrieval from slave to master is 
> something like a GET and that information injection into the master conn is 
> something like a POST. So, effectively (and not necessarily in HTTP wire 
> format), a slave connection would to a
> 
> GET /ssl/StdEnvVars
> 
> to retrieve the map. Which would, the first time, get added to the incoming 
> queue on the master thread, be answered, slave thread awoken again and 
> processing continues. This would be cached for future queries, so that slaves 
> can access it without being put to sleep.
> 
> We could use our famous HOOK infrastructure so that modules can register 
> their own handlers for such information. We'd only need to nail down the 
> general semantics and the data format for interchange. Something like a const 
> apr_table_t would probably work. It would need to be readonly and allow 
> concurrent use.
> 
> The problem I encountered in mod_ftp was unbounded memory consumption in the 
> control connection (similar schema to mod_http2 master/child)...
> 
> In the mod_ftp case, it was login credentials. A unique pool off of the 
> connection that could be cleared on a new login was necessary to keep us from 
> consuming more and more pool on each login reattempt.
> 
> Same will happen here, that apr_table_t of ssl envvars will need to survive 
> until all requests running off of that list of connection strings are 
> complete, and then released. Maybe a simple apr_atomic_inc32/dec32 would 
> suffice for a ref count, based on a cleanup registered on the request pool? 
> In any case, the request should use the result table it requested, and not 
> some shared pointer in the conn_rec.

Yes. And my idea would be to keep the "cached" values at the slave connection. 
mod_http2 keeps a spare slave connection around, but surplus slaves are 
destroyed. That should eliminate a memory buildup on long-lived connections.

Re: concurrent ssl context access

[William A Rowe Jr]

On Tue, Jul 12, 2016 at 3:37 AM, Stefan Eissing <
stefan.eiss...@greenbytes.de> wrote:

>
> > Am 11.07.2016 um 17:38 schrieb William A Rowe Jr :
> >
> > On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing <
> stefan.eiss...@greenbytes.de> wrote:
> > In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept
> up that StdEnvVars makes concurrent access to the SSL context when HTTP/2
> is in place.
> >
> > The question is: how do we want to fix this?
> >
> > The general theme is that a module needs to perform some action on a
> connection, sees that it is a slave connection and accesses the master
> instead. Such a module needs to be aware that access to master may happen
> in parallel with other request.
> >
> > Should we let each such module care for it on its own or do we want to
> provide a way to access the master connection in a mutex protected way?
> >
> > Rather than a mutex, which would become an issue because all master
> access
> > to the context would require blocking, is there a way we can allow a
> module to
> > perform its (hopefully swift) processing on the master thread? Send a
> metadata
> > bucket to the master asking for a function to be invoked?
>
> Hmm, interesting idea. Of course, there would still be a mutex involved in
> handling such a query, but maybe it can be done more elegant than mutexing
> all accesses to structures everywhere.
>
> One way to look at this is that information retrieval from slave to master
> is something like a GET and that information injection into the master conn
> is something like a POST. So, effectively (and not necessarily in HTTP wire
> format), a slave connection would to a
>
> GET /ssl/StdEnvVars
>
> to retrieve the map. Which would, the first time, get added to the
> incoming queue on the master thread, be answered, slave thread awoken again
> and processing continues. This would be cached for future queries, so that
> slaves can access it without being put to sleep.
>
> We could use our famous HOOK infrastructure so that modules can register
> their own handlers for such information. We'd only need to nail down the
> general semantics and the data format for interchange. Something like a
> const apr_table_t would probably work. It would need to be readonly and
> allow concurrent use.
>

The problem I encountered in mod_ftp was unbounded memory consumption in
the control connection (similar schema to mod_http2 master/child)...

In the mod_ftp case, it was login credentials. A unique pool off of the
connection that could be cleared on a new login was necessary to keep us
from consuming more and more pool on each login reattempt.

Same will happen here, that apr_table_t of ssl envvars will need to survive
until all requests running off of that list of connection strings are
complete, and then released. Maybe a simple apr_atomic_inc32/dec32 would
suffice for a ref count, based on a cleanup registered on the request pool?
In any case, the request should use the result table it requested, and not
some shared pointer in the conn_rec.

Re: mod_proxy_fcgi, 304 responses and bogus error messages

[Luca Toscano]

2016-07-11 23:33 GMT+02:00 Luca Toscano :

>
>
> 2016-07-11 19:44 GMT+02:00 Jacob Champion :
>
>> On 07/11/2016 08:53 AM, Luca Toscano wrote:
>>
>>> I am looking for some feedback about the patch proposed to figure out if
>>> I am on the right track or not. Does it make sense to read all the data
>>> returned by a FCGI backend even on error conditions to avoid subsequent
>>> spurious reads or is there a smarter method?
>>>
>>
>> (following up from IRC)
>>
>> Regarding your patch: I think that, rather than duplicate the jump back
>> to recv_again, the error handling logic around the call to
>> ap_scan_script_header_* should be improved. That function is documented to
>> return binary success/failure (OK/INTERNAL_SERVER_ERROR), but in reality it
>> can return other things too -- definitely NOT_MODIFIED (which is handled),
>> perhaps PRECONDITION_FAILED (which is not handled), and maybe others?
>>
>

I followed up on the 412 use case (HTTP_PRECONDITION_FAILED) trying to
reproduce it. mod_proxy_fcgi logs correctly a 412 in the access log (as
opposed to a 503 for the HTTP_NOT_MODIFIED use case) plus the following:

[proxy_fcgi:error] [pid 12325:tid 139974709171968] [client ::1:56696]
AH01070: Error parsing script headers
[proxy_fcgi:error] (22)Invalid argument: [client ::1:56696] AH01075: Error
dispatching request to :

It looks a bit confusing to me, if we want to remove them we might threat a
412 like a 304 (so no body sent to the client, no break in the code since
there might be some more remaining bytes to read).

Luca

Re: concurrent ssl context access

[Stefan Eissing]

> Am 11.07.2016 um 17:38 schrieb William A Rowe Jr :
> 
> On Mon, Jul 11, 2016 at 5:25 AM, Stefan Eissing 
>  wrote:
> In https://bz.apache.org/bugzilla/show_bug.cgi?id=59840 the issue crept up 
> that StdEnvVars makes concurrent access to the SSL context when HTTP/2 is in 
> place.
> 
> The question is: how do we want to fix this?
> 
> The general theme is that a module needs to perform some action on a 
> connection, sees that it is a slave connection and accesses the master 
> instead. Such a module needs to be aware that access to master may happen in 
> parallel with other request.
> 
> Should we let each such module care for it on its own or do we want to 
> provide a way to access the master connection in a mutex protected way?
> 
> Rather than a mutex, which would become an issue because all master access
> to the context would require blocking, is there a way we can allow a module to
> perform its (hopefully swift) processing on the master thread? Send a metadata
> bucket to the master asking for a function to be invoked?

Hmm, interesting idea. Of course, there would still be a mutex involved in 
handling such a query, but maybe it can be done more elegant than mutexing all 
accesses to structures everywhere.

One way to look at this is that information retrieval from slave to master is 
something like a GET and that information injection into the master conn is 
something like a POST. So, effectively (and not necessarily in HTTP wire 
format), a slave connection would to a

GET /ssl/StdEnvVars

to retrieve the map. Which would, the first time, get added to the incoming 
queue on the master thread, be answered, slave thread awoken again and 
processing continues. This would be cached for future queries, so that slaves 
can access it without being put to sleep.

We could use our famous HOOK infrastructure so that modules can register their 
own handlers for such information. We'd only need to nail down the general 
semantics and the data format for interchange. Something like a const 
apr_table_t would probably work. It would need to be readonly and allow 
concurrent use.

-Stefan