Need information about Apache and Module development.
Dear All, We are planning for an enhancement of our servers.We are looking out if we write Apache module and use Apache to support the below requirements. We would like to use Apache as a routing layer that accepts both structured documents (XML docs) also some other predefined data blocks in addition to key/value paris via http and then routes those transactions based on a key available in the request such that the required keys and sub keys are all routed to the same rules instance - this will require the original request to be broadcast to multiple instances of the rules engine ( for example, one instance may be tracking the IP, another instance may be tracking by GUID or Screenname, and yet another may be tracking by some Signature.) A rules execution engine that is able to accept the request, evaluate a set of ops defined rules and execute various responses. There are preference of using DROOLS fusion.Any functionality in Apache based on Rules ? What are the approaches need to be taken for dynamic load balancing.Like suppose I have 3 instances of Apache is running and due to some issue one of the instance goes down.I would expect the traffic should be balanced properly by the existing 2 instances. For load balancing apart from mod_proxy_balancer any other Apache modules can be worth looking into? As and alert broadcast engine that has the ability to distribute an events to multiple end sources. As a storage layer that allows for data persistence for long term tracking of keys and values. We have the target of good performance.Is it going to be good using Apache or any other webserver would be suggestion. Inputs would be very much appreciated. Many thanks in advance -A
Re: blocking Upgrade
On Mar 29, 2011, at 11:16 PM, Greg Stein wrote: Do you have an internet draft spec for some context here? Is there a proposal for HTTP/2.0? websockets I might also argue that a directive is not the right answer here. Instead, I'd suggest that modules advertise their ability to consume protocols. If an Upgrade arrives, and a relevant module is found, then the request is handled. If no such module is found, then the Upgrade header is ignored, and nothing happens.˛˛ If a module or CGI sees the Upgrade header, then what is the problem? It cannot truly alter the protocol in effect for the connection. That seems to be something only possible to handle within the core (to change the connection handler). And back to the AllowUpgrade directive. What the heck should it do if something is present *besides* none. It isn't like that can be handled. Some code must be written to provide other protocols, and *that* code should manage the tokens recognized. Not a directive. Right, I woke up thinking the same thing -- that the http filter is the only place that an Upgrade could actually be implemented because it would have to switch itself off. What I was trying to avoid is a script thinking that it can switch without writing a new protocol filter, or a CGI that deliberately spoofs an upgrade for some nefarious purpose I haven't thought of yet. We can just delete the Upgrade header in the http_filter, for now. Roy
Re: blocking Upgrade
On 30 Mar 2011, at 10:49 AM, Roy T. Fielding wrote: On Mar 29, 2011, at 11:16 PM, Greg Stein wrote: Do you have an internet draft spec for some context here? Is there a proposal for HTTP/2.0? websockets In theory, over and above natively supporting websockets, it may be useful to teach mod_proxy how to reverse proxy the ws scheme to another websockets server. Would a module like mod_proxy_websockets or mod_proxy_ws make sense? Regards, Graham --
Re: Need information about Apache module development.
dev@httpd.apache.org is for discussions related to development of httpd itself. Your questions below are more appropriate for the Third Party Module Authors' List. See http://httpd.apache.org/lists.html#modules-dev A rules execution engine that is able to accept the request, evaluate a set of ops defined rules and execute various responses. There are preference of using DROOLS fusion.Any functionality in Apache based on Rules ? Most Apache HTTP Server configuration directives can be thought of as rules. But this is probably not very helpful. Note that Drools is written in Java, while Apache HTTP Server is written in C. If you want to use Drools, you may want to consider using a web server that is written in Java. What are the approaches need to be taken for dynamic load balancing.Like suppose I have 3 instances of Apache is running and due to some issue one of the instance goes down.I would expect the traffic should be balanced properly by the existing 2 instances. This will not happen unless you have a load balancer that is external to Apache HTTP Server. For load balancing apart from mod_proxy_balancer any other Apache modules can be worth looking into? mod_proxy_balancer runs on a front-end (proxy) server to balance requests that the front-end server receives between multiple back-end servers. If one of the back-end servers goes down, the front-end server will detect this and split the traffic betweent the remaining back-end servers. However, this may not be what you want. As and alert broadcast engine that has the ability to distribute an events to multiple end sources. Apache HTTP Server is not a broadcast engine. This is functionality that you would have to write and include in your module. As a storage layer that allows for data persistence for long term tracking of keys and values. We have the target of good performance.Is it going to be good using Apache or any other webserver would be suggestion. Apache HTTP Server is not a storage layer. Apache HTTP Server just processes HTTP requests. In order to process these requests, Apache HTTP Server normally serves files from some filesystem or other storage layer which is external to Apache HTTP Server itself. (Normally, this would be some local filesystem such as ext3 or NTFS; but it could also be a remote or distributed filesystem such as NFS, CIFS, or AFS. In turn, the remote filesystem could be based on iSCSI, FibreChannel, or other technologies.) Inputs would be very much appreciated. Many thanks in advance Good luck. -- Mark Montague m...@catseye.org
A timestamp for mod_log_forensic (?)
Hi there, Mod_log_forensic is saving my day while debugging a crashing apache. But matching the right request with the crash and its corefile is difficult. Ideally the log would show me only the active requests at the moment the server died. But in my case things are a bit more difficult. The delta between incoming requests and those finished is bigger. So I matched the entries of the finished requests with the access log entries to get a more or less accurate timestamp for all those requests that never finished, so I could match them with the crash. But that is very complicated of course. So, is there anything speaking against a timestamp for the forensic log? The format right now looks as follows +yQtJf8CoAB4AAFNXBIEA|GET /manual/de/images/down.gif HTTP/1.1|Host:localhost%3a8080| etc. A format with a microtimestamp could look as follows: +956166333.123456|yQtJf8CoAB4AAFNXBIEA|GET /manual/de/ ... or +yQtJf8CoAB4AAFNXBIEA|956166333.123456|GET /manual/de/ ... or +yQtJf8CoAB4AAFNXBIEA|GET /manual/de/ ... |956166333.123456| Best regards, Christian Folini -- Christian Folini - christian.fol...@netnea.com
Re: A timestamp for mod_log_forensic (?)
On 30 Mar 2011, at 3:23 PM, Christian Folini wrote: Mod_log_forensic is saving my day while debugging a crashing apache. But matching the right request with the crash and its corefile is difficult. Have you taken a look at Jeff's mod_whatkilledus? http://people.apache.org/~trawick/exception_hook.html Regards, Graham --
Re: Need information about Apache and Module development.
On 30 Mar 2011, at 13:59, Arnab Ganguly wrote: Dear All, You've posted this three times now, and it's still no clearer what you want. Either you're implementing something yourself, or you want to hire someone to do a job for you. If the latter then please say so! If the former, you need to figure out the problem and ask clear questions. I could infer half a dozen possible questions from your post, but an answer to any of them is unlikely to help in the absence of clarity. We are planning for an enhancement of our servers.We are looking out if we write Apache module and use Apache to support the below requirements. Yes you can. If your application runs on HTTP then it might even be a good solution. We would like to use Apache This looks like the beginning of a job spec. One to discuss with your developer, who may then ask more specific questions here when particular issues arise. -- Nick Kew Available for work, contract or permanent http://www.webthing.com/~nick/cv.html
Re: blocking Upgrade
On Mar 30, 2011, at 12:32 PM, Graham Leggett wrote: On 30 Mar 2011, at 10:49 AM, Roy T. Fielding wrote: On Mar 29, 2011, at 11:16 PM, Greg Stein wrote: Do you have an internet draft spec for some context here? Is there a proposal for HTTP/2.0? websockets In theory, over and above natively supporting websockets, it may be useful to teach mod_proxy how to reverse proxy the ws scheme to another websockets server. Would a module like mod_proxy_websockets or mod_proxy_ws make sense? No, websockets is not designed to work with intermediaries. There is no standard behavior beyond opening the connection, so connections through proxies should use CONNECT. I am not planning to implement websockets -- I just want to protect the rest of the server from kids playing with toys. Roy
Re: blocking Upgrade
On 30 Mar 2011, at 3:53 PM, Roy T. Fielding wrote: No, websockets is not designed to work with intermediaries. There is no standard behavior beyond opening the connection, so connections through proxies should use CONNECT. Does a websocket client have a way of knowing it should use CONNECT? Regards, Graham --
Handling APR_HAS_LARGE_FILES
Hello, we are trying to develop a proprietary (i.e. closed-source, binary-shipped) Apache module and have run into troubles with binary incompatibilities because of APR_HAS_LARGE_FILES being either 0 or 1. I have only found http://osdir.com/ml/apache.apr.devel/2003-05/msg00028.html but this does not exactly describe our problem. The problem is that depending on APR_HAS_LARGE_FILES the sizes and offsets of structs (e.g. request_rec) change, breaking binary compatibility. Now we could compile the module twice - once with large file support and once without large file support but this leads to yet more binaries we would have to ship (32/64bit, Apache2.0/2.2, LFS=0/LFS=1). Besides, I guess most customers do not know/care if their httpd was compiled with LFS or without. So my question: How do you handle these kinds of problems? (Are there any proprietary Apache modules out there? ;-)) Are there some best practices or is there something I am missing? Is there a way to find out at run-time if httpd was compiled with LFS or without? Thanks, Ignaz
Re: blocking Upgrade
On Mar 30, 2011, at 4:11 PM, Graham Leggett wrote: On 30 Mar 2011, at 3:53 PM, Roy T. Fielding wrote: No, websockets is not designed to work with intermediaries. There is no standard behavior beyond opening the connection, so connections through proxies should use CONNECT. Does a websocket client have a way of knowing it should use CONNECT? My guess is that it would if it were told to use a proxy for ws. Keep in mind that when I say proxy, I do not mean to include reverse proxy. A reverse proxy of websockets is just an implementation of websockets or a tunnel. I consider both to be pretty dangerous and better done in a special-purpose server rather than as a child of Apache httpd. Roy
Re: Handling APR_HAS_LARGE_FILES
On Wed, Mar 30, 2011 at 16:25, Ignaz Birnstingl ign...@gmail.com wrote: we are trying to develop a proprietary (i.e. closed-source, binary-shipped) Apache module and have run into troubles with binary incompatibilities because of APR_HAS_LARGE_FILES being either 0 or 1. I have only found http://osdir.com/ml/apache.apr.devel/2003-05/msg00028.html but this does not exactly describe our problem. The problem is that depending on APR_HAS_LARGE_FILES the sizes and offsets of structs (e.g. request_rec) change, breaking binary compatibility. Now we could compile the module twice - once with large file support and once without large file support but this leads to yet more binaries we would have to ship (32/64bit, Apache2.0/2.2, LFS=0/LFS=1). Besides, I guess most customers do not know/care if their httpd was compiled with LFS or without. So my question: How do you handle these kinds of problems? (Are there any proprietary Apache modules out there? ;-)) Are there some best practices or is there something I am missing? Is there a way to find out at run-time if httpd was compiled with LFS or without? If you have to ship a binary blob, you are probably best off shipping a module for each supported platform and supply some glue source code that loads the appropriate module at runtime, akin to what the NVIDIA Linux driver does. But if you are going to support a multitude of platforms and architectures, you really should reconsider your 'binary only' policy. I assure you support is going to be a royal pain, not to mention the irate customers. This is the voice of experience speaking.
Re: blocking Upgrade
On 30 Mar 2011, at 4:41 PM, Roy T. Fielding wrote: My guess is that it would if it were told to use a proxy for ws. Keep in mind that when I say proxy, I do not mean to include reverse proxy. A reverse proxy of websockets is just an implementation of websockets or a tunnel. I consider both to be pretty dangerous and better done in a special-purpose server rather than as a child of Apache httpd. When you say pretty dangerous, are you referring to the danger of reaching the limit of slots willing to be served by the server, or something else? I would say that any server that supports the idea of long lived connections faces this danger, I don't see why httpd (suitably and sensibly configured obviously, probably with an event mpm and a limit as to the number of slots we allow to be long lived) can't play too. Regards, Graham --
Re: blocking Upgrade
On Wed, Mar 30, 2011 at 11:08, Graham Leggett minf...@sharp.fm wrote: On 30 Mar 2011, at 4:41 PM, Roy T. Fielding wrote: My guess is that it would if it were told to use a proxy for ws. Keep in mind that when I say proxy, I do not mean to include reverse proxy. A reverse proxy of websockets is just an implementation of websockets or a tunnel. I consider both to be pretty dangerous and better done in a special-purpose server rather than as a child of Apache httpd. When you say pretty dangerous, are you referring to the danger of reaching the limit of slots willing to be served by the server, or something else? I would say that any server that supports the idea of long lived connections faces this danger, I don't see why httpd (suitably and sensibly configured obviously, probably with an event mpm and a limit as to the number of slots we allow to be long lived) can't play too. I think that Roy's point is simply that httpd would be nothing more than a socket-listener and tunnel. There is very little that it can bring to the table at that point, so it doesn't make a lot of sense to lump in websockets capabilities. Cheers, -g
Re: blocking Upgrade
On 30 Mar 2011, at 5:48 PM, Greg Stein wrote: I think that Roy's point is simply that httpd would be nothing more than a socket-listener and tunnel. There is very little that it can bring to the table at that point, so it doesn't make a lot of sense to lump in websockets capabilities. In theory, for somebody it might be a socket listener and a tunnel, using the same HTTP endpoint as their existing HTTP server, with the same port and the same digital certificate, perhaps additionally secured by an authn or authz module. While I don't doubt that for a gateway like this to be practical there would be issues to overcome, such as ensuring long lived requests cannot starve out other server requests, however I do see some value in trying it. Disclaimer: ENOTIME right now, so no plans to do so for a while. Regards, Graham --
Re: A timestamp for mod_log_forensic (?)
On Wed, Mar 30, 2011 at 03:32:27PM +0200, Graham Leggett wrote: Have you taken a look at Jeff's mod_whatkilledus? http://people.apache.org/~trawick/exception_hook.html mod_whatkilledus will be one of the next steps in my debugging, if mod_forensic won't do. Still I think I could add a small timestamp patch for mod_log_forensic for future convenience. regs, Christian -- Christian Folini - christian.fol...@netnea.com
slotmem_plain slotmem consumers
Hello, It seems that slotmem_plain is not being found by the consumers which then causes a configuration error. Attached patch fixes this and makes the error message a little more friendlier for users by hinting to load slotmem which is currently not documented as a requirement of these modules. Regards, Gregg slotmem_consumers_fix.diff Description: Binary data
mod_watchdog crashes Apache on Windows
Hello, After r1070317 a block of code got moved below the block of Win32 debug code that starts now at line 447. The code that was above it is now just directly below but this has caused a crash on Windows for whatever reason (still unknown to me). The attached patch moves this block of debugging code down where it should be to correspond to where it used to be pre r1070317. The other alternative is to remove the block of debugging code completely should someone feel that might be more appropriate. Regards, Gregg watchdog_win32_fix.diff Description: Binary data
