Re: HTTP proxy working for folks on 2.1-dev?
Jeff Trawick wrote: Just committed the needed changes to allow forward proxies. I saw... definitely gets farther now... This shows how far it gets when I configure mozilla to use Apache as HTTP proxy: [Thu Sep 09 06:53:17 2004] [crit] [Thu Sep 09 06:53:17 2004] file http_protocol.c, line 981, assertion "readbytes > 0" failed [Thu Sep 09 06:53:18 2004] [notice] child pid 4606 exit signal Abort (6), possible coredump in /export/home/trawick/inst/21 Yes, I've just tried mozilla too, and it core dumps :(. Interesting when using IE as a client everything works. I know where the problem is. Give me couple of hours to test that on each mpm. Regards, MT. smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
On Thu, 9 Sep 2004, Mladen Turk wrote: > Q: > Is it possible to have forward and reverse proxies mixed together > on the same box? Of course! I have that defined in different virtual hosts, but AFIACS it should also work fine simply using for the reverse proxies and for the forward. -- Nick Kew
Re: HTTP proxy working for folks on 2.1-dev?
On Thu, 09 Sep 2004 12:52:53 +0200, Mladen Turk <[EMAIL PROTECTED]> wrote: > > > Jeff Trawick wrote: > > On Fri, 3 Sep 2004 12:30:34 -0400, Jeff Trawick <[EMAIL PROTECTED]> wrote: > >>> > > 192.168.1.11 - - [03/Sep/2004:12:05:59 -0400] "GET > http://127.0.0.1:10101/cgi-bin/printenv HTTP/1.0" 404 236 > > error log has: > > [Fri Sep 03 12:05:59 2004] [error] [client 127.0.0.1] File does not > exist: proxy:http://127.0.0.1:10101/cgi-bin/printenv > >> > > I had time dig into it enough to get the feeling that it is something > > that the balancer/worker folks ought to have a look at ;) It would be > > a big headstart knowing what is supposed to happen in the handler > > hook. See attached function call trace. Does the balancer's handler > > have to return OK? Does the balancer's proxy-pre_request hook have to > > return OK? > > Just committed the needed changes to allow forward proxies. I saw... definitely gets farther now... This shows how far it gets when I configure mozilla to use Apache as HTTP proxy: [Thu Sep 09 06:53:16 2004] [debug] mod_proxy.c(654): Trying to run scheme_handler [Thu Sep 09 06:53:16 2004] [debug] proxy_http.c(1195): proxy: HTTP: serving URL http://planetsun.org/ [Thu Sep 09 06:53:16 2004] [debug] proxy_util.c(1483): proxy: initialized worker for (*:0) min=0 max=25 smax=25 [Thu Sep 09 06:53:16 2004] [debug] proxy_util.c(1414): proxy: socket is constructed [Thu Sep 09 06:53:16 2004] [debug] proxy_util.c(1586): proxy: HTTP: has acquired connection for (*:0) [Thu Sep 09 06:53:16 2004] [debug] proxy_util.c(1640): proxy: connecting http://planetsun.org/ to planetsun.org:80 [Thu Sep 09 06:53:16 2004] [debug] proxy_util.c(1789): proxy: HTTP: fam 2 socket created to connect to *:0 [Thu Sep 09 06:53:16 2004] [debug] proxy_util.c(1880): proxy: HTTP: connection complete to 194.70.142.72:80 (planetsun.org) [Thu Sep 09 06:53:17 2004] [debug] proxy_http.c(1016): proxy: start body send [Thu Sep 09 06:53:17 2004] [crit] [Thu Sep 09 06:53:17 2004] file http_protocol.c, line 981, assertion "readbytes > 0" failed [Thu Sep 09 06:53:18 2004] [notice] child pid 4606 exit signal Abort (6), possible coredump in /export/home/trawick/inst/21 (gdb) where #0 0xd116200c in _lwp_kill () from /lib/libc.so.1 #1 0xd115f24d in thr_kill () from /lib/libc.so.1 #2 0xd110c7af in raise () from /lib/libc.so.1 #3 0xd10eef34 in abort () from /lib/libc.so.1 #4 0x080cf256 in ap_log_assert (szExp=0x80fa235 "readbytes > 0", szFile=0x80fa0f0 "http_protocol.c", nLine=981) at log.c:708 #5 0x08091d86 in ap_http_filter (f=0x823dff8, b=0x8232e58, mode=AP_MODE_READBYTES, block=APR_BLOCK_READ, readbytes=0) at http_protocol.c:981 #6 0x080da031 in ap_get_brigade (next=0x823dff8, bb=0x8232e58, mode=AP_MODE_READBYTES, block=APR_BLOCK_READ, readbytes=0) at util_filter.c:474 #7 0x080e3c46 in net_time_filter (f=0x8233e30, b=0x8232e58, mode=AP_MODE_READBYTES, block=APR_BLOCK_READ, readbytes=0) at core.c:3768 #8 0x080da031 in ap_get_brigade (next=0x8233e30, bb=0x8232e58, mode=AP_MODE_READBYTES, block=APR_BLOCK_READ, readbytes=0) at util_filter.c:474 #9 0x0808d4e9 in ap_proxy_http_process_response (p=0x8231f78, r=0x8237fc8, backend=0x81e7e30, origin=0x82326b8, conf=0x8180598, server_portstr=0xcf66dd20 ":8080") at proxy_http.c:1027 #10 0x0808da6f in ap_proxy_http_handler (r=0x8237fc8, worker=0x8186490, conf=0x8180598, url=0x8232658 "/", proxyname=0x0, proxyport=0) at proxy_http.c:1254 #11 0x080828cf in proxy_run_scheme_handler (r=0x8237fc8, worker=0x8186490, conf=0x8180598, url=0x8239216 "http://planetsun.org/";, proxyhost=0x0, proxyport=0) at mod_proxy.c:1749 #12 0x08080419 in proxy_handler (r=0x8237fc8) at mod_proxy.c:656 #13 0x080cabb5 in ap_run_handler (r=0x8237fc8) at config.c:156 #14 0x080cb312 in ap_invoke_handler (r=0x8237fc8) at config.c:368 #15 0x08095864 in ap_process_request (r=0x8237fc8) at http_request.c:246 #16 0x0808fb2e in ap_process_http_connection (c=0x82320a0) at http_core.c:253 #17 0x080d719a in ap_run_process_connection (c=0x82320a0) at connection.c:42 #18 0x080d7586 in ap_process_connection (c=0x82320a0, csd=0x8231fb0) at connection.c:175 #19 0x080c7305 in process_socket (p=0x8231f78, sock=0x8231fb0, my_child_num=1, my_thread_num=24, bucket_alloc=0x8235f88) at worker.c:520 #20 0x080c7a96 in worker_thread (thd=0x81e4418, dummy=0x817e650) at worker.c:856 > > Q: > Is it possible to have forward and reverse proxies mixed together > on the same box? definitely
Re: HTTP proxy working for folks on 2.1-dev?
Jeff Trawick wrote: On Fri, 3 Sep 2004 12:30:34 -0400, Jeff Trawick <[EMAIL PROTECTED]> wrote: 192.168.1.11 - - [03/Sep/2004:12:05:59 -0400] "GET http://127.0.0.1:10101/cgi-bin/printenv HTTP/1.0" 404 236 error log has: [Fri Sep 03 12:05:59 2004] [error] [client 127.0.0.1] File does not exist: proxy:http://127.0.0.1:10101/cgi-bin/printenv I had time dig into it enough to get the feeling that it is something that the balancer/worker folks ought to have a look at ;) It would be a big headstart knowing what is supposed to happen in the handler hook. See attached function call trace. Does the balancer's handler have to return OK? Does the balancer's proxy-pre_request hook have to return OK? Just committed the needed changes to allow forward proxies. Q: Is it possible to have forward and reverse proxies mixed together on the same box? For example: The httpd is acting as forward proxy, but also has defined few reverse proxies. In that case instead default worker we could directly use reverse proxies workers together with balancing etc... Regards, MT. smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
Graham Leggett wrote: Mladen Turk wrote: This is a forward proxy? Are you speaking about that? If do, then the current implementation might be bogus, cause frankly speaking didn't test that a lot, but was planning to do so. Or I've missed the subject again :). "Proxyrequest on" turns httpd into a forward proxy, yes. OK. I'll add something like a 'default worker' for that. Perhaps even add some circular buffer to speed up the hostname resolving. Not sure if the balancer can help in case of forward proxies. If you think it can, some use case will help a lot. Regards, MT. smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
Mladen Turk wrote: This is a forward proxy? Are you speaking about that? If do, then the current implementation might be bogus, cause frankly speaking didn't test that a lot, but was planning to do so. Or I've missed the subject again :). "Proxyrequest on" turns httpd into a forward proxy, yes. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
Jeff Trawick wrote: Can you explain some real world usage of such configuration. Configure your web browser to use Apache as an HTTP proxy. This is a forward proxy? Are you speaking about that? If do, then the current implementation might be bogus, cause frankly speaking didn't test that a lot, but was planning to do so. Or I've missed the subject again :). Regards, MT. smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
On Wed, 08 Sep 2004 17:49:56 +0200, Mladen Turk <[EMAIL PROTECTED]> wrote: > Can you explain some real world usage of such configuration. Configure your web browser to use Apache as an HTTP proxy. > P.S. > Seems I've missed a day in school when they talk about > ProxyVia headers :). Forget about ProxyVia for now, as that is just some metadata handling which takes place if we get the HTTP connection handling fixed. When/if you care about ProxyVia, this has all you need to know: http://httpd.apache.org/docs-2.0/mod/mod_proxy.html#proxyvia
Re: HTTP proxy working for folks on 2.1-dev?
Graham Leggett wrote: It is not a problem to add the worker dynamically if never specified, but I'm worried about the security in that case. Is the NoProxy meant to be used in such situations? This is why there was a split between determining the one (or more) IP addresses to potentially connect to, and then deciding which of those addresses is the best to connect to based on load balancer / round robin / whatever - sometime the downstream servers are client specified, rather than preconfigured in a config. Here, the balancer might say "You are asking proxy to connect to an IP address that balancer has no clue about in my config. Let me just DNS round robin this one". OK, we'll need to add the new worker if the DNS resolves the valid connection point/protocol. Some Q's: 1. How can I suppress such a behavior if not needed 2. Is this a default behavior no mater what the config says 3. How to calculate the loadfactors when the new node is resolved 4. What about cases when I can connect to the node but the node returns 404 cause there is no application requested on that node, or I'm missing something. The final ones: Can you explain some real world usage of such configuration. Can it be done using standard directives and what would they look like. P.S. Seems I've missed a day in school when they talk about ProxyVia headers :). Regards, MT. smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
Mladen Turk wrote: OK, but still what are the remotes to connect to? If they are dynamically determined then I'll need to fix the way the workers are determined. Right now each physical box we can connect to has to be at least once specified in the config. The question is: Is it OK to resolve that on the first successful request on the unspecified host? It is not a problem to add the worker dynamically if never specified, but I'm worried about the security in that case. Is the NoProxy meant to be used in such situations? This is why there was a split between determining the one (or more) IP addresses to potentially connect to, and then deciding which of those addresses is the best to connect to based on load balancer / round robin / whatever - sometime the downstream servers are client specified, rather than preconfigured in a config. Here, the balancer might say "You are asking proxy to connect to an IP address that balancer has no clue about in my config. Let me just DNS round robin this one". Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
Graham Leggett wrote: proxyrequests on proxyvia on allowconnect 8081 80 8080 OK. From that config what would be remotes that we could connect to? Dynamically obtained from Via header or...? Seems to me that the balancer is totally unusable in such a configuration. Keep in mind "proxyrequests on" enables forward proxy operation ala squid. Here the balancer could be used to choose between the best downstream website and/or proxy. OK, but still what are the remotes to connect to? If they are dynamically determined then I'll need to fix the way the workers are determined. Right now each physical box we can connect to has to be at least once specified in the config. The question is: Is it OK to resolve that on the first successful request on the unspecified host? It is not a problem to add the worker dynamically if never specified, but I'm worried about the security in that case. Is the NoProxy meant to be used in such situations? Regards, MT. smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
Mladen Turk wrote: This is a very basic proxy config: proxyrequests on proxyvia on allowconnect 8081 80 8080 OK. From that config what would be remotes that we could connect to? Dynamically obtained from Via header or...? Seems to me that the balancer is totally unusable in such a configuration. Keep in mind "proxyrequests on" enables forward proxy operation ala squid. Here the balancer could be used to choose between the best downstream website and/or proxy. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
Jeff Trawick wrote: What is the config that you are using. Does you requests get passed with previous version of proxy. If they do, please post the config so we can find why is it breaking. This is a very basic proxy config: proxyrequests on proxyvia on allowconnect 8081 80 8080 OK. From that config what would be remotes that we could connect to? Dynamically obtained from Via header or...? Seems to me that the balancer is totally unusable in such a configuration. Regards, MT. smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
On Wed, 08 Sep 2004 08:39:43 +0200, Mladen Turk <[EMAIL PROTECTED]> wrote: >If the balancer is not found (the uri doesn't start with >proxy:balancer://) then the each particular scheme handler is called. Scheme handlers aren't getting called since ap_proxy_pre_request() returns something other than OK, causing proxy_handler() to return prematurely. > What is the config that you are using. Does you requests get > passed with previous version of proxy. If they do, please post the > config so we can find why is it breaking. This is a very basic proxy config: proxyrequests on proxyvia on allowconnect 8081 80 8080 It fails as described above with main branch (2.1-dev) HEAD; it works fine with APACHE_2_0_BRANCH HEAD.
Re: HTTP proxy working for folks on 2.1-dev?
Jeff Trawick wrote: [Fri Sep 03 12:05:59 2004] [error] [client 127.0.0.1] File does not exist: proxy:http://127.0.0.1:10101/cgi-bin/printenv If nobody can/has reproduced the problem, I'll dig into it this weekend. I had time dig into it enough to get the feeling that it is something that the balancer/worker folks ought to have a look at ;) It would be a big headstart knowing what is supposed to happen in the handler hook. See attached function call trace. Does the balancer's handler have to return OK? Does the balancer's proxy-pre_request hook have to return OK? Balancer handler returns OK only if you set something like: SetHandler balancer-manager It is used for dynamic balancer manager (enabling/disabling members, changing load factors, etc...). So, just like any handler (status, info, ...) it should return DECLINED. pre_request hook returns OK only if the balancer is found. Looking in the trace you've provided, it behaves just as it should, cause it seems that you didn't define any balancer in the config, so none is found and DECLINED is returned. If the balancer is not found (the uri doesn't start with proxy:balancer://) then the each particular scheme handler is called. What is the config that you are using. Does you requests get passed with previous version of proxy. If they do, please post the config so we can find why is it breaking. Regards, MT. smime.p7s Description: S/MIME Cryptographic Signature
Re: HTTP proxy working for folks on 2.1-dev?
On Fri, 3 Sep 2004 12:30:34 -0400, Jeff Trawick <[EMAIL PROTECTED]> wrote: > > > On Fri, 03 Sep 2004 09:23:27 -0700, Justin Erenkrantz > <[EMAIL PROTECTED]> wrote: > > --On Friday, September 3, 2004 12:14 PM -0400 Jeff Trawick <[EMAIL PROTECTED]> > > wrote: > > > > > > > > > I'm using head, with a spelling fix to mod_proxy comment (probably the > > > cause of the breakage) and a tweak to allow proxy connect to bypass > > > the balancer, and this silly testcase isn't working: > > > > > > 192.168.1.11 - - [03/Sep/2004:12:05:59 -0400] "GET > > > http://127.0.0.1:10101/cgi-bin/printenv HTTP/1.0" 404 236 > > > > > > error log has: > > > > > > [Fri Sep 03 12:05:59 2004] [error] [client 127.0.0.1] File does not > > > exist: proxy:http://127.0.0.1:10101/cgi-bin/printenv > > If nobody can/has reproduced the problem, I'll dig into it this weekend. I had time dig into it enough to get the feeling that it is something that the balancer/worker folks ought to have a look at ;) It would be a big headstart knowing what is supposed to happen in the handler hook. See attached function call trace. Does the balancer's handler have to return OK? Does the balancer's proxy-pre_request hook have to return OK? handler_trace Description: Binary data
Re: HTTP proxy working for folks on 2.1-dev?
--On Friday, September 3, 2004 12:14 PM -0400 Jeff Trawick <[EMAIL PROTECTED]> wrote: I'm using head, with a spelling fix to mod_proxy comment (probably the cause of the breakage) and a tweak to allow proxy connect to bypass the balancer, and this silly testcase isn't working: 192.168.1.11 - - [03/Sep/2004:12:05:59 -0400] "GET http://127.0.0.1:10101/cgi-bin/printenv HTTP/1.0" 404 236 error log has: [Fri Sep 03 12:05:59 2004] [error] [client 127.0.0.1] File does not exist: proxy:http://127.0.0.1:10101/cgi-bin/printenv Sounds like the proxy_http module isn't picking up on the request. -- justin
