Re: mod_ssl namespacing: app_data2
On Fri, May 1, 2015 at 8:58 AM, Stefan Sperling wrote: >... > So modssl_ has been agreed on? :) > It hasn't been agreed. Just not denied. Yet. :-P
Re: Proposal/RFC: "informed" load balancing
I’m Cc: this to d...@trafficserver.apache.org, since I think this is something some of our dev would be interested in. There are a few other replies to this thread already, which can be seen on the archives. As has been mentioned in another reply, I think the header name ought to be something ^X- (see RFC 6648). Backend-Info or Backend-Capacity-Info or some such? Cheers, — leif > On Apr 29, 2015, at 10:54 PM, Jim Riggs wrote: > > [ Long message and proposal follows. Bear with me. There are a lot of words, > but that is because we need a lot of help/input! ;-) ] > > So, this has come up in the past several times, and we discussed it again > this year at ApacheCon: How do we get the load balancer to make smarter, more > informed decisions about where to send traffic? > > The different LB methods provide some different attempts at balancing > traffic, but ultimately none of them is "smart" about its decision. Other > than a member being in error state, the balancer makes its decision solely > based on configuration (LB set, factor, etc.) and its own knowledge of the > member (e.g. requests, bytes). What we have often discussed is a way to get > some type of health/load/capacity information from the backend to make > informed balancing decisions. > > One method is to use health checks (a la haproxy, AWS ELBs, etc.) that > request one or more URLs and the response code/time indicates whether or not > the service is up and available, allowing more proactive decisions. While > this is better than our current state of reactively marking members in error > state based on failed requests, it still provides a limited view of the > health/state of the backend. > > We have also discussed implementing a way for backends to communicate a > magical "load" number to the front end to take into account as it balances > traffic. This would give a much better view into the backend's state, but > requires some way to come up with this calculation that each backend > system/server/service/app must provide. This then has to be implemented in > all the various backends (e.g. httpd, tomcat, php-fpm, unicorn, mongrel, > etc., etc.), probably a hard sell to all of those projects. And, the front > end would have limited control over what that number means or how to use it. > > During JimJag's balancer talk at ApacheCon this year, he brought up this > issue of "better, more informed" decision making again. I put some thought > into it that night and came up with some ideas. Jim, Covener, Trawick, > Ruggeri, and I then spent some time over the next couple of days talking it > through and fleshing out some of the details. > > Based on all of that, below is what I am proposing. I have some initial code > that I am working on to implement the different pieces of this, and I will > put them up in bugz or somewhere when they're a little less rudimentary. > > -- > > Our hope is to create a general standard that can be used by various > projects, products, proxies, servers, etc., to have a more consistent way for > a load balancer to request and receive useful internal state information from > its backend nodes. This information can then be used by the *frontend* > software/admin (this is the main change from what we have discussed before) > to calculate a load factor appropriate for each backend node. > > This communication uses a new, standard HTTP header, "X-Backend-Info", that > takes this form in RFC2616 BNF: > >backend-info = "version" "=" numeric-entry >[ > *LWS "," *LWS > #( numeric-entry | string-entry ) >] > >numeric-entry = numeric-field "=" ( float | <"> float <"> ) >; that is, numbers may optionally be enclosed in >; quotation marks > >float = 1*DIGIT [ "." 1*DIGIT ] > >numeric-field = "workers-max" >; maximum number of workers the backend supports > | "workers-used" >; current number of used/busy workers > | "workers-allocated" >; current number of allocated/ready workers > | "workers-free" >; current number of workers available for use >; (generally the difference between workers-max and >; workers-used, though some implementations may have >; a different notion) > | "uptime" >; number of seconds the backend has been running > | "requests" >; number of requests the backend has processed > | "memory-max" >; total amount of memory available in bytes > | "memory-used" >; amount of used memory in bytes > | "memory-allocated" >; amount of allocated/committed memor
Re: *Match, RewriteRule POLA violation?
> On 1 May 2015, at 10:52, André Malo wrote: > > * Niklas Edmundsson wrote: > >> On Thu, 30 Apr 2015, Yann Ylavic wrote: >>> On Thu, Apr 30, 2015 at 2:57 PM, Jim Riggs > wrote: Thanks, Yann. I remember looking at this code before. The question remains, though: Is it currently "wrong"? Does it need to be "fixed", or was this distinction made intentionally? Is there a specific use case that requires the regex-matching directives to not get slash-normalized URIs? >>> >>> I would like it to be fixed, non leading "/+" is equivalent to "/", >>> this would break very few (if any) cases IMHO, and may even unbreak >>> more ones . >> >> +1 >> >> I would expect Location and LocationMatch using the same uri for >> comparison. > > Hmm, that assumption is wrong by definition. Location always matches a > prefix (a part of a parsed/unparsed url), while LocationMatch always > matches the complete URL. We need to be careful on that phrasing. LocationMatch *can* match the complete URL if it is anchored at both ends. By default, though, it will match anywhere in the URL if it is unanchored, and it can (with the '/+' we are discussing here) behave just like Location and match a prefix if it is only anchored at the beginning. >> I would actually go so far as the current state might >> warrant a CVE for being a hidden security risk that might cause >> inadvertent information exposure. > > It *is* documented right here, btw: > http://httpd.apache.org/docs/2.4/mod/core.html#location > > (found it, eventually...) Yay! I knew I had read it somewhere. Good find! However, that documentation is actually not correct. This is not true: "For example, would match the request URL /abc but not the request URL //abc." Based on all of my tests, the leading slash *is* collapsed in the *Match and RewriteRule directives. Subsequent slashes after the first are not. So, it is documented, but is there a compelling use case for this? When would someone actually need to match against the multiple slashes (unless it's some really strange hack someone has implemented)? The only thing I can think of is that maybe you want to force a redirect to remove multiple slashes, but couldn't a directive in mod_alias(?) maybe handle that if deemed necessary? Also, yes it is documented, but when I brought it up at a hackathon table full of long-time committers at ApacheCon, no one knew about it, and I had to make an example. This says to me says that maybe it's unexpected behavior and a potential security risk as both Niklas and Ruggeri have suggested. That's why I brought it up in the first place. And so the questions still remain: Right or wrong? Change it or leave it? Expected or unexpected? Security risk or user/configuration error? The world may never know... ;-) I may go ahead and write up a patch this weekend to change them all (*Match and RewriteRule) and then we can all debate it over on bugz too!
Re: Proposal/RFC: "informed" load balancing
On 1 May 2015, at 01:30, Daniel Ruggeri wrote: >> >> 4. The backend MUST add the "X-Backend-Info" token to the "Connection" >> response header, making it a hop-by-hop field that is removed by the >> frontend from the downstream response (RFC2616 14.10 and RFC7230 6.1). [Note >> there appears to be an httpd bug here that I intend to submit and that needs >> to be addressed.] >> >>Connection: X-Backend-Info > > I'm not sure if this is a stroke of brilliance or extra work that isn't > needed :-) . As we discussed at the Con, it is vital for the proxy to remove > the header to avoid leaking any potentially useful information to an attacker > out to the 'tubes... but parsing Connection for "X-Backend-Info" seems like > it wouldn't be needed since one could just as well check if X-Backend-Info > header is present. I'm probably missing the obvious, but can you help me > understand more about why we would want this here instead of treating the > presence of the header as a sign to do some kind of work? Here's a situation that could go wrong if this new header weren't marked as hop-by-hop. Imagine if there are two webserver products in a reverse proxy topology, something like this: user-agent ← httpd-proxy ← acme-proxy ← httpd-origin (the server tiers might in fact be clusters of identically configured hosts). All 4 tiers are doing HTTP/1.1 cacheing, correctly using Vary: and so on. If httpd-origin is sending X-Backend-Info then it must signal to ACME-proxy that this is a hop-by-hop header. Let's say httpd-origin signals that workers-free is 0. httpd-proxy receives a copy of this header and from acme-proxy. httpd-proxy incorrectly concludes that workers-free is 0 and starts sending 503 responses as per its intended configuration, even though acme-proxy would be able to serve stale responses from its cache. The sysadmin contacts the vendor “ACME Proxy”; the vendor asserts that their product is conforming to HTTP 1.1 and that the incorrect behaviour is in Apache httpd. Which, in my view, it would be. -- Tim Bannister – is...@c8h10n4o2.org.uk
Re: Looking ahead to 2.4.13 / 2.2.30
On 4/30/15 2:52 PM, William A Rowe Jr wrote: > It seems that we have 2 groups of good things to come out of ApacheCon, > some immediate fixes for things like BSD project efforts, some pretty > straightforward defects that have been resolved... and then there's a bunch > of energy about enhancements and the h2 universe. > > In the short term, WDYT about giving the trees a week to settle, grab the > low hanging fruit and move forward for 2.4.13 and 2.2.30 end of this coming > week? Guessing Jim's up for RM'ing 2.4.13, and I'm happy to T&R 2.2.30 > in tandem. > > Concerns / observations / thoughts? I have a mod_dav fix that really ought to be in the next 2.4 release. I'll get it committed and nominated sometime this weekend.
Re: Proposal/RFC: "informed" load balancing
On 1 May 2015, at 01:30, Daniel Ruggeri wrote: > > On 4/29/2015 11:54 PM, Jim Riggs wrote: >> >> So, this has come up in the past several times, and we discussed it again >> this year at ApacheCon: How do we get the load balancer to make smarter, >> more informed decisions about where to send traffic? … >>string-entry = string-field "=" ( token | quoted-string ) > > A useful token could be status=OK|ERROR|MAINTENANCE where a backend could > advertise to the upstream load balancer that it may want to be put in drain > mode or something to that effect. Since this list can't/won't be exhaustive > of all things people could care to send, let's add some head room in the spec > by allowing "custom-integer" and/or "custom-string". Otherwise, I suspect > people would cram things into the wrong fields just to get the data back to > the proxy. Although I think it's an approach deprecated by IETF, how about allowing any field name provided it's prefixed with “x-”? -- Tim Bannister – is...@c8h10n4o2.org.uk
Re: svn commit: r1677149 - in /httpd/httpd/trunk/modules/ssl: ssl_util_ssl.c ssl_util_ssl.h
On Sat, May 02, 2015 at 11:10:50AM +0200, Kaspar Brand wrote: > On 01.05.2015 16:29, s...@apache.org wrote: > > Author: stsp > > Date: Fri May 1 14:28:59 2015 > > New Revision: 1677149 > > > > URL: http://svn.apache.org/r1677149 > > Log: > > mod_ssl namespacing: Make SSL_ASN1_STRING_to_utf8 a static function inside > > ssl_util_ssl.c (no callers outside this file). The new static function name > > chosen is convert_asn1_to_utf8, based on the assumption that neither SSL_ > > nor ASN1_ are safe prefixes to use without potential future overlap. > > Thanks for pushing ahead with the namespace cleanup, Stefan. I'm fine > with making the function static, but would suggest to rename it to > "asn1_string_to_utf8" instead - first, because it's really limited to > ASN.1 strings (doesn't deal with arbitrary ASN.1 data), and second, > because the "_to_" already implies a conversion (plus it matches the > naming of the other comparable functions in ssl_util_ssl.c). > > Kaspar Done: http://svn.apache.org/r1677339 I had minor reservations about using the asn1_ prefix since it is used inside of OpenSSL (though not in public APIs) and it is used in public APIs by GNU Libtasn1, used by GnuTLS. But it seems unnecessary to avoid a clash with asn1_ because mod_ssl's is a static function and mod_ssl won't compile with GnuTLS anyway.
Re: svn commit: r1677149 - in /httpd/httpd/trunk/modules/ssl: ssl_util_ssl.c ssl_util_ssl.h
On 01.05.2015 16:29, s...@apache.org wrote: > Author: stsp > Date: Fri May 1 14:28:59 2015 > New Revision: 1677149 > > URL: http://svn.apache.org/r1677149 > Log: > mod_ssl namespacing: Make SSL_ASN1_STRING_to_utf8 a static function inside > ssl_util_ssl.c (no callers outside this file). The new static function name > chosen is convert_asn1_to_utf8, based on the assumption that neither SSL_ > nor ASN1_ are safe prefixes to use without potential future overlap. Thanks for pushing ahead with the namespace cleanup, Stefan. I'm fine with making the function static, but would suggest to rename it to "asn1_string_to_utf8" instead - first, because it's really limited to ASN.1 strings (doesn't deal with arbitrary ASN.1 data), and second, because the "_to_" already implies a conversion (plus it matches the naming of the other comparable functions in ssl_util_ssl.c). Kaspar
Re: mod_ssl: inline SSL_X509_INFO_load_path(); please review
On 01.05.2015 17:11, Stefan Sperling wrote: > I believe SSL_X509_INFO_load_path() should be inlined into > its only caller. I'm +1 for this. The "Low-Level CA Certificate Loading" part in ssl_util_ssl.c is / was only used by ssl_init_proxy_certs, so I would be in favor of also moving SSL_X509_INFO_load_file to ssl_engine_init.c (and making it static). > Regarding the removed comment about merging the dir-read loop > with another one: I don't think that's worth it. If we can get rid of code duplication on this occasion, then I think we should do so - it makes future maintenance easier if there is common code for loading CA files from a directory, be that for client authentication (SSLCACertificatePath) or for proxy connections (SSLProxyCACertificatePath). Kaspar
