Re: mod_proxy_fcgi issues
I still DO have PHP commit! :) On Dec 4, 2014, at 1:27 PM, Eric Covener cove...@gmail.com wrote: On Thu, Dec 4, 2014 at 1:11 PM, Jim Riggs apache-li...@riggs.me wrote: This all may certainly be true, but I just for clarity's sake (since it was my quote that started this new mod_proxy_fcgi thread), my mod_proxy_balancer - mod_proxy_fcgi - php-fpm issue is NOT an httpd issue...at least that is not how I have treated it. It is actually a code fix I have had to make in PHP to get it to work. See: https://bugs.php.net/bug.php?id=54152 https://bugs.php.net/bug.php?id=62172 It would be great if JimJag could maybe address this if he still has PHP access. Jim? It was motivated by you reply but not that report of the issue (also bouncing around in httpd bugzilla). It doesn't seem that usable values for these things should be so unique to php-fpm.
Re: mod_proxy_fcgi issues
On Fri, Dec 5, 2014 at 9:23 PM, Jeff Trawick traw...@gmail.com wrote: I guess PATH_INFO is the issue that affects the most people; thoughts? I guess expression-based support for overriding the default PATH_INFO calculation is the basis for solving the widest variety of problems with any of the variables, though I don't know if we'll have the right string operations available (e.g., substring of dynamic length); thoughts? I think this is still missing from string-returning expressions because you can't run a regex and return the backref. sf id say a long time ago when I asked about this that a module could run a boolean expression to set the backrefs, then reuse some kind of ap_expr context and still have the backrefs available (I've never actually looked at this part) My use case back then was something like spitting out part of a variable as the value in Header set foo expr= You can call single-argument functions that returns a string, and I believe that single argument is interpolated by ap_expr so it could be powerful. A function could then have its own delimeter to work around the problem with args. This is only worth discussing to save the work/risk in changing the expression parser itself. The function would not need to be fcgi specific, it would just be like e.g. PHP's preg_replace() or somehow in an operator (that isn't =~) -- Eric Covener cove...@gmail.com
Re: mod_proxy_fcgi issues
On Sat, Dec 6, 2014 at 3:29 PM, Eric Covener cove...@gmail.com wrote:
You can call single-argument functions that returns a string, and I
believe that single argument is interpolated by ap_expr so it could be
powerful. A function could then have its own delimeter to work
around the problem with args. This is only worth discussing to save
the work/risk in changing the expression parser itself. The function
would not need to be fcgi specific, it would just be like e.g. PHP's
preg_replace() or somehow in an operator (that isn't =~)
How about an ap_expr's function to help on this? (like Jan's
replace(), but for regexps)
Something like: extract([subject], [capture(s)], [substitution]):
= substitute(%{PATH_INFO}, ^(.*)/strippedpathinfo(/.*)/notpathinfo, $1$2).
so to extract all is captured by [capture(s)] and play with it (them)
in [substitution].
Or: substitute([subject], [pattern], [substitution]):
= substitute(%{PATH_INFO}, ^(.*)/notpathinfo.*, $1).
so to substitue all is matched by [pattern] with [substitution].
(hence the trailing .* is needed here).
Just a matter of taste...
Regards,
Yann.
Re: mod_proxy_fcgi issues
On Sat, Dec 6, 2014 at 10:11 AM, Yann Ylavic ylavic@gmail.com wrote:
On Sat, Dec 6, 2014 at 3:29 PM, Eric Covener cove...@gmail.com wrote:
You can call single-argument functions that returns a string, and I
believe that single argument is interpolated by ap_expr so it could be
powerful. A function could then have its own delimeter to work
around the problem with args. This is only worth discussing to save
the work/risk in changing the expression parser itself. The function
would not need to be fcgi specific, it would just be like e.g. PHP's
preg_replace() or somehow in an operator (that isn't =~)
How about an ap_expr's function to help on this? (like Jan's
replace(), but for regexps)
Something like: extract([subject], [capture(s)], [substitution]):
= substitute(%{PATH_INFO}, ^(.*)/strippedpathinfo(/.*)/notpathinfo,
$1$2).
so to extract all is captured by [capture(s)] and play with it (them)
in [substitution].
Or: substitute([subject], [pattern], [substitution]):
= substitute(%{PATH_INFO}, ^(.*)/notpathinfo.*, $1).
so to substitue all is matched by [pattern] with [substitution].
(hence the trailing .* is needed here).
Just a matter of taste...
Regards,
Yann.
I think this is good, but needs one of a few things for places where
we want strings as the result.
- start at word instead of string in the grammar
- make function calls directly accessible from string
- make multiple args in %{funcname} style calls
- shoehorn 3 args into 1 that doesn't upset the current parser/scanner
--
Eric Covener
cove...@gmail.com
Re: mod_proxy_fcgi issues
On Thu, Dec 4, 2014 at 11:50 AM, Jeff Trawick traw...@gmail.com wrote: On Thu, Dec 4, 2014 at 10:38 AM, Eric Covener cove...@gmail.com wrote: forked from apachecon thread On Thu, Dec 4, 2014 at 10:23 AM, Jeff Trawick traw...@gmail.com wrote: On Thu, Dec 4, 2014 at 9:58 AM, Eric Covener cove...@gmail.com wrote: On Tue, Dec 2, 2014 at 4:14 PM, Jim Riggs apache-li...@riggs.me wrote: P.S. mod_proxy_balancer - mod_proxy_fcgi - php-fpm is really fun and interesting too! ;-) mod_proxy_fcgi seems to need a bit of work from what I have been seeing in bugzilla and IRC. I hope to spend a little time on the code and doc, but not being an actual user of it I don't know how far I will really get before being distracted. This is very important stuff IMO. I know we don't do the coordination thing around here, but if the work was organized to some extent, perhaps 3-4 people could easily share the work??? (bite sized chunks of the development: simple reproducers, doc, code, review, whatever) Besides searching through Bugzilla and summarizing each mod_proxy_fcgi bug and ranking by apparent severity, number of users involved in the bug discussion, etc., what else should I put on a Wiki page? E.g., do you have an idea of what needs to be improved in the doc? I don't have much more than what amounts to the summary/initial poking around in the bugzillas. I had the wiki thought too -- bullets there are much easie than trying to reabsorb the PRs each time. - examples need to account for php-fpm (how URLs and or paths are passed) - fixes for CGI variables in different configurations (sethandler vs. proxypass) -- fixup r-filename right before adding CGI vars, maybe directory walk -- path info calculation probably needs multiple modes. Maybe expr based? FWIW, every time I see the PATH_INFO questions I recall the nginx configuration (the script vars provided are those defined in the configuration, with some expression support); we need general CGISetVar var expr that sets or overrides script variables in common code (like CGIPassAuth) so that it works with the many modules that interface with scripts. - further doc for worker matching stuff with ProxySet - provide a convenience/less verbose directive to configure SetHandler + a backend worker -- doc SetHandler advantages - change compile-time diag stuff to trace8 - need to port mod_proxy_http or mod_fcgid body spooling / content length passing - would be nice to have a non php-fpm fastcgi server to sanity check with so we don't end up with too many php-fpm-isms - figure out / make sure balancer examples work with php-fpm and/or setHandler And although mod_proxy_fcgi is the usual suspect, some of these are important issues with mod_proxy_scgi too. (It is faster, and for some deployments there's no natural preference for FastCGI over SCGI.) Thanks for all the notes; I'll write this stuff up today. Initial version at https://wiki.apache.org/httpd/Development/mod_proxy_fcgi I haven't integrated your list into the table yet -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: mod_proxy_fcgi issues
On Fri, Dec 5, 2014 at 7:45 AM, Jeff Trawick traw...@gmail.com wrote: On Thu, Dec 4, 2014 at 11:50 AM, Jeff Trawick traw...@gmail.com wrote: On Thu, Dec 4, 2014 at 10:38 AM, Eric Covener cove...@gmail.com wrote: forked from apachecon thread On Thu, Dec 4, 2014 at 10:23 AM, Jeff Trawick traw...@gmail.com wrote: On Thu, Dec 4, 2014 at 9:58 AM, Eric Covener cove...@gmail.com wrote: On Tue, Dec 2, 2014 at 4:14 PM, Jim Riggs apache-li...@riggs.me wrote: P.S. mod_proxy_balancer - mod_proxy_fcgi - php-fpm is really fun and interesting too! ;-) mod_proxy_fcgi seems to need a bit of work from what I have been seeing in bugzilla and IRC. I hope to spend a little time on the code and doc, but not being an actual user of it I don't know how far I will really get before being distracted. This is very important stuff IMO. I know we don't do the coordination thing around here, but if the work was organized to some extent, perhaps 3-4 people could easily share the work??? (bite sized chunks of the development: simple reproducers, doc, code, review, whatever) Besides searching through Bugzilla and summarizing each mod_proxy_fcgi bug and ranking by apparent severity, number of users involved in the bug discussion, etc., what else should I put on a Wiki page? E.g., do you have an idea of what needs to be improved in the doc? I don't have much more than what amounts to the summary/initial poking around in the bugzillas. I had the wiki thought too -- bullets there are much easie than trying to reabsorb the PRs each time. - examples need to account for php-fpm (how URLs and or paths are passed) - fixes for CGI variables in different configurations (sethandler vs. proxypass) -- fixup r-filename right before adding CGI vars, maybe directory walk -- path info calculation probably needs multiple modes. Maybe expr based? FWIW, every time I see the PATH_INFO questions I recall the nginx configuration (the script vars provided are those defined in the configuration, with some expression support); we need general CGISetVar var expr that sets or overrides script variables in common code (like CGIPassAuth) so that it works with the many modules that interface with scripts. - further doc for worker matching stuff with ProxySet - provide a convenience/less verbose directive to configure SetHandler + a backend worker -- doc SetHandler advantages - change compile-time diag stuff to trace8 - need to port mod_proxy_http or mod_fcgid body spooling / content length passing - would be nice to have a non php-fpm fastcgi server to sanity check with so we don't end up with too many php-fpm-isms - figure out / make sure balancer examples work with php-fpm and/or setHandler And although mod_proxy_fcgi is the usual suspect, some of these are important issues with mod_proxy_scgi too. (It is faster, and for some deployments there's no natural preference for FastCGI over SCGI.) Thanks for all the notes; I'll write this stuff up today. Initial version at https://wiki.apache.org/httpd/Development/mod_proxy_fcgi I haven't integrated your list into the table yet Now that's cleaned up. I guess PATH_INFO is the issue that affects the most people; thoughts? I guess expression-based support for overriding the default PATH_INFO calculation is the basis for solving the widest variety of problems with any of the variables, though I don't know if we'll have the right string operations available (e.g., substring of dynamic length); thoughts?
Re: mod_proxy_fcgi issues
On Thu, Dec 4, 2014 at 10:38 AM, Eric Covener cove...@gmail.com wrote: forked from apachecon thread On Thu, Dec 4, 2014 at 10:23 AM, Jeff Trawick traw...@gmail.com wrote: On Thu, Dec 4, 2014 at 9:58 AM, Eric Covener cove...@gmail.com wrote: On Tue, Dec 2, 2014 at 4:14 PM, Jim Riggs apache-li...@riggs.me wrote: P.S. mod_proxy_balancer - mod_proxy_fcgi - php-fpm is really fun and interesting too! ;-) mod_proxy_fcgi seems to need a bit of work from what I have been seeing in bugzilla and IRC. I hope to spend a little time on the code and doc, but not being an actual user of it I don't know how far I will really get before being distracted. This is very important stuff IMO. I know we don't do the coordination thing around here, but if the work was organized to some extent, perhaps 3-4 people could easily share the work??? (bite sized chunks of the development: simple reproducers, doc, code, review, whatever) Besides searching through Bugzilla and summarizing each mod_proxy_fcgi bug and ranking by apparent severity, number of users involved in the bug discussion, etc., what else should I put on a Wiki page? E.g., do you have an idea of what needs to be improved in the doc? I don't have much more than what amounts to the summary/initial poking around in the bugzillas. I had the wiki thought too -- bullets there are much easie than trying to reabsorb the PRs each time. - examples need to account for php-fpm (how URLs and or paths are passed) - fixes for CGI variables in different configurations (sethandler vs. proxypass) -- fixup r-filename right before adding CGI vars, maybe directory walk -- path info calculation probably needs multiple modes. Maybe expr based? FWIW, every time I see the PATH_INFO questions I recall the nginx configuration (the script vars provided are those defined in the configuration, with some expression support); we need general CGISetVar var expr that sets or overrides script variables in common code (like CGIPassAuth) so that it works with the many modules that interface with scripts. - further doc for worker matching stuff with ProxySet - provide a convenience/less verbose directive to configure SetHandler + a backend worker -- doc SetHandler advantages - change compile-time diag stuff to trace8 - need to port mod_proxy_http or mod_fcgid body spooling / content length passing - would be nice to have a non php-fpm fastcgi server to sanity check with so we don't end up with too many php-fpm-isms - figure out / make sure balancer examples work with php-fpm and/or setHandler And although mod_proxy_fcgi is the usual suspect, some of these are important issues with mod_proxy_scgi too. (It is faster, and for some deployments there's no natural preference for FastCGI over SCGI.) Thanks for all the notes; I'll write this stuff up today. -- Born in Roswell... married an alien... http://emptyhammock.com/
Re: mod_proxy_fcgi issues
On Thu, Dec 4, 2014 at 9:58 AM, Eric Covener cove...@gmail.com wrote: On Tue, Dec 2, 2014 at 4:14 PM, Jim Riggs apache-li...@riggs.me wrote: P.S. mod_proxy_balancer - mod_proxy_fcgi - php-fpm is really fun and interesting too! ;-) mod_proxy_fcgi seems to need a bit of work from what I have been seeing in bugzilla and IRC. I hope to spend a little time on the code and doc, but not being an actual user of it I don't know how far I will really get before being distracted. This all may certainly be true, but I just for clarity's sake (since it was my quote that started this new mod_proxy_fcgi thread), my mod_proxy_balancer - mod_proxy_fcgi - php-fpm issue is NOT an httpd issue...at least that is not how I have treated it. It is actually a code fix I have had to make in PHP to get it to work. See: https://bugs.php.net/bug.php?id=54152 https://bugs.php.net/bug.php?id=62172 It would be great if JimJag could maybe address this if he still has PHP access. Jim?
Re: mod_proxy_fcgi issues
On Thu, Dec 4, 2014 at 1:11 PM, Jim Riggs apache-li...@riggs.me wrote: This all may certainly be true, but I just for clarity's sake (since it was my quote that started this new mod_proxy_fcgi thread), my mod_proxy_balancer - mod_proxy_fcgi - php-fpm issue is NOT an httpd issue...at least that is not how I have treated it. It is actually a code fix I have had to make in PHP to get it to work. See: https://bugs.php.net/bug.php?id=54152 https://bugs.php.net/bug.php?id=62172 It would be great if JimJag could maybe address this if he still has PHP access. Jim? It was motivated by you reply but not that report of the issue (also bouncing around in httpd bugzilla). It doesn't seem that usable values for these things should be so unique to php-fpm.
Re: mod_proxy_fcgi issues
On 2014-12-04 13:27, Eric Covener wrote:
On Thu, Dec 4, 2014 at 1:11 PM, Jim Riggs apache-li...@riggs.me wrote:
This all may certainly be true, but I just for clarity's sake (since it was my quote
that started this new mod_proxy_fcgi thread), my mod_proxy_balancer -
mod_proxy_fcgi - php-fpm issue is NOT an httpd issue...at least that is not how I
have treated it. It is actually a code fix I have had to make in PHP to get it to
work.
[...] It doesn't seem that usable values for these things should be so unique
to php-fpm.
My experience has been that the PHP FPM SAPI function
init_request_info() in sapi/fpm/fpm/fpm_main.c, which I think was
originally copied from the CGI SAPI, is very old code that goes to great
lengths to preserve old, not always standards-compliant behavior in
order to avoid breaking backward compatibilities. Hence, I'm not
convinced that the things Eric refers to above might not be unique to
php-fpm.
After struggling to get php-fpm working with mod_proxy_fcgi, I
eventually completely rewrote the whole init_request_info function the
way I thought it should be without any regards to backwards
compatibility; this solved the problems I was having.
If memory serves (it's been a few years) the main problems I was
encountering were with serving the index file for directories and
correct handling of PATH_INFO.
I've attached the patch I'm using (a completely new version of the
init_request_info function) in case anyone wants to either play with it
or compare it to the code that PHP currently uses.
--
Mark Montague
m...@catseye.org
diff -up php-5.6.3/sapi/fpm/fpm/fastcgi.c.fpm-init-request
php-5.6.3/sapi/fpm/fpm/fastcgi.c
--- php-5.6.3/sapi/fpm/fpm/fastcgi.c.fpm-init-request 2014-11-18
20:33:20.313769152 +
+++ php-5.6.3/sapi/fpm/fpm/fastcgi.c2014-11-18 20:33:38.424369147 +
@@ -488,6 +488,7 @@ static int fcgi_get_params(fcgi_request
ret = 0;
break;
}
+zlog(ZLOG_DEBUG, fcgi_get_params: %s=%s, tmp, s);
zend_hash_update(req-env, tmp, eff_name_len+1, s,
sizeof(char*), NULL);
p += name_len + val_len;
}
@@ -1093,12 +1094,14 @@ char* fcgi_putenv(fcgi_request *req, cha
{
if (var req) {
if (val == NULL) {
+ zlog(ZLOG_DEBUG, fcgi_putenv: %s=, var);
zend_hash_del(req-env, var, var_len+1);
} else {
char **ret;
val = estrdup(val);
if (zend_hash_update(req-env, var, var_len+1, val,
sizeof(char*), (void**)ret) == SUCCESS) {
+ zlog(ZLOG_DEBUG, fcgi_putenv: %s=%s, var,
val);
return *ret;
}
}
diff -up php-5.6.3/sapi/fpm/fpm/fpm_main.c.fpm-init-request
php-5.6.3/sapi/fpm/fpm/fpm_main.c
--- php-5.6.3/sapi/fpm/fpm/fpm_main.c.fpm-init-request 2014-11-12
13:52:21.0 +
+++ php-5.6.3/sapi/fpm/fpm/fpm_main.c 2014-11-18 20:33:38.425369123 +
@@ -1422,6 +1422,317 @@ static void init_request_info(TSRMLS_D)
}
/* }}} */
+static char *fpm_cgibin_saveenv(char *name, char *val)
+{
+int name_len = strlen(name);
+char *old_val = sapi_cgibin_getenv(name, name_len TSRMLS_CC);
+char save_name[256];
+
+if (val != NULL old_val != NULL strcmp(val, old_val) == 0) {
+ return old_val;
+}
+
+if (name_len 256 - strlen(ORIG_) - 1) {
+ strcpy(save_name, ORIG_);
+ strcat(save_name, name);
+} else {
+ save_name[0] = '\0';
+ }
+
+/* Save the old value only if one was not previously saved */
+if (old_val save_name[0] != '\0'
+ sapi_cgibin_getenv(save_name, strlen(save_name) TSRMLS_CC) ==
NULL) {
+ _sapi_cgibin_putenv(save_name, old_val TSRMLS_CC);
+ }
+
+ return _sapi_cgibin_putenv(name, val TSRMLS_CC);
+
+}
+
+static void init_request_info0(TSRMLS_D)
+{
+char *document_root;
+int document_root_len;
+char *script_filename;
+int script_filename_len;
+char *script_filename_part = NULL;
+char *s = NULL;
+char *path = NULL;
+char *path_info = NULL;
+char *path_translated = NULL;
+char *content_type;
+char *content_length;
+const char *auth;
+char *ini;
+int result;
+struct stat st;
+int add_index = 0;
+
+zlog(ZLOG_DEBUG, initializing request info:);
+
+/* initialize the defaults */
+SG(request_info).path_translated = NULL;
+SG(request_info).request_method = NULL;
+SG(request_info).proto_num = 1000;
+SG(request_info).query_string = NULL;
+SG(request_info).request_uri = NULL;
+SG(request_info).content_type = NULL;
+SG(request_info).content_length = 0;
+SG(sapi_headers).http_response_code = 200;
+
+/*
+ * Use our document root instead of one passed to us by our invoker.
+
