Re: Dynamic updates and Apache v2.0
Hi, On Die 27.07.2004 19:45, Mladen Turk wrote: B. Backend reports I'm going down, no more new sessions please If these can be setup in the httpd with an option for a virtserver that would be nice ;-)) (add Shutdown 30Min graceful restart) But i think here is it offtopic, is'n it? al ;-) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Graham Leggett wrote: Costin Manolache wrote: Hard == replicating the configuration data to all the nodes, instead of having it in a central place ( file or a config server ). Not impossible, but it's a different problem, and not very commonly used Ldap, nis, ldap and most other config servers are using the later. The admin might choose to populate the server list on just the seed tomcat server, instead of all the tomcat servers. proxy_ajp would have to handle this situation. Or tomcat might use the session sharing mechanism to spread the info on all the servers in the pool. The idea is to use knowledge where we already have it - for example, if we're sharing sessions between servers, that mechanism already has the list of servers in the cluster, so we're using info we already have. We're also not expecting the admin to give us info we can work out by ourselves if it can be avoided, thus adding to the admin's workload. I agree. When a new server is added to the pool, the admin should only have to configure the new server - and it should be able to register itself with the seed or master config. BTW, there are quite a few protocols related with discovery ( zeroconf, etc ), but I'm not sure people would be comfortable with this as a default. In the ideal case, 95% of users should just be able to do this: ProxyPass /myWebapp ajp://cluster-seed/myWebapp And the cluster should be autodetected and handled for the admin. Ok. ( with 2-3 comma separated seed servers ? ) Costin Regards, Graham -- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Costin Manolache wrote: Graham Leggett wrote: Costin Manolache wrote: Hard == replicating the configuration data to all the nodes, instead of having it in a central place ( file or a config server ). Not impossible, but it's a different problem, and not very commonly used Ldap, nis, ldap and most other config servers are using the later. The admin might choose to populate the server list on just the seed tomcat server, instead of all the tomcat servers. proxy_ajp would have to handle this situation. Or tomcat might use the session sharing mechanism to spread the info on all the servers in the pool. The idea is to use knowledge where we already have it - for example, if we're sharing sessions between servers, that mechanism already has the list of servers in the cluster, so we're using info we already have. We're also not expecting the admin to give us info we can work out by ourselves if it can be avoided, thus adding to the admin's workload. I agree. When a new server is added to the pool, the admin should only have to configure the new server - and it should be able to register itself with the seed or master config. +1 :-) BTW, there are quite a few protocols related with discovery ( zeroconf, etc ), but I'm not sure people would be comfortable with this as a default. In the ideal case, 95% of users should just be able to do this: ProxyPass /myWebapp ajp://cluster-seed/myWebapp And the cluster should be autodetected and handled for the admin. Ok. ( with 2-3 comma separated seed servers ? ) Graham suggested something like this : ProxyPass /myWebapp ajp://cluster-seed/myWebapp ajp://cluster-seed2/myWebapp ajp://cluster-seed3/myWebapp So we're in a situation where the URI mapping is hardcoded in httpd.conf but where the pool tomcat to handle such mapping could be updated dynamically ? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Graham Leggett wrote: Henri Gomez wrote: Well using an URL could be a good idea, since it could be : - a static file, edited by admin, on the web-server or another web-server/tomcat - a dynamically generated file, PHP/JSP/Servlet/PERL, whatever. An URL is a logical solution, which leaves us with two choices: - A specific out-of-band URL predefined in httpd where httpd is told get your config regularly here. This URL could be served by tomcat, or any other server. - In-band hop-by-hop header information that is deployed with responses from the backend servers. This has the advantage of having httpd learn about the status of backends on an up to the minute basis on each hit. httpd will already use certain in-band info, like whether a request was successful or not, or how long the request took. So if someone breaks in one node, he has the entire pool :-) I don't know - I like to know where the configuration is and to have it under some control. I understand having it in a file on the httpd server is not what people want, but still - having each worker in the pool push config on arbitrary requests seems a bit extreme, and much harder to implement on the server side as well. Costin I prefer the in-band approach, as it's consistent. As has been pointed out, BEA Weblogic does this with success, so admins out there will not find this a foreign concept. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Costin Manolache wrote: I just hope you agree that having cluster info in httpd.conf is the worst possible solution - you have all the control access problems, you need to gracefully restart often, it is very hard to automate and use tools to manage it, etc. I 100% agree with this statement, yes. Regards, Graham -- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Costin Manolache wrote: So if someone breaks in one node, he has the entire pool :-) If someone breaks into one node, then they very likely have access to any backend databases anyway. Being able to manipulate the pool is the least of your worries. But it's a valid concern nonetheless. I don't know - I like to know where the configuration is and to have it under some control. Some control is a valid point. Being able to switch on and off the dynamic features would be useful if it was in httpd.conf (the home of site policy information - if our site doesn't want to allow dynamic adding of servers to the pool, it's set once in httpd.conf and that's it). I understand having it in a file on the httpd server is not what people want, but still - having each worker in the pool push config on arbitrary requests seems a bit extreme, and much harder to implement on the server side as well. Depends on what you mean by hard. I don't see anything hard about adding headers to a response, it is something we do already. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic updates and Apache v2.0
Graham Leggett wrote: is not what people want, but still - having each worker in the pool push config on arbitrary requests seems a bit extreme, and much harder to implement on the server side as well. Depends on what you mean by hard. I don't see anything hard about adding headers to a response, it is something we do already. Hard == replicating the configuration data to all the nodes, instead of having it in a central place ( file or a config server ). Not impossible, but it's a different problem, and not very commonly used Ldap, nis, ldap and most other config servers are using the later. Costin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Costin Manolache wrote: Hard == replicating the configuration data to all the nodes, instead of having it in a central place ( file or a config server ). Not impossible, but it's a different problem, and not very commonly used Ldap, nis, ldap and most other config servers are using the later. The admin might choose to populate the server list on just the seed tomcat server, instead of all the tomcat servers. proxy_ajp would have to handle this situation. Or tomcat might use the session sharing mechanism to spread the info on all the servers in the pool. The idea is to use knowledge where we already have it - for example, if we're sharing sessions between servers, that mechanism already has the list of servers in the cluster, so we're using info we already have. We're also not expecting the admin to give us info we can work out by ourselves if it can be avoided, thus adding to the admin's workload. In the ideal case, 95% of users should just be able to do this: ProxyPass /myWebapp ajp://cluster-seed/myWebapp And the cluster should be autodetected and handled for the admin. If the admin has special needs, or would like to turn off specific features like dynamic updates, then the admin is free to use directives inside httpd.conf to do this to override the default behaviour. The point is to keep the default typical case as simple as humanly possible. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Re: Dynamic updates and Apache v2.0
Thanks for your email. This is an auto response to confirm we have received your email. If you are wishing to check the whereabouts of an order please use our order tracker at - http://www.seetickets.com/tracker have your reference number and email address handy! If you wish to change your delivery address, we have an on-line address change facility. Please send a blank e-mail to [EMAIL PROTECTED] with your reference number and the last four digits of the card used to place the order in the subject line. Many thanks for booking with See Tickets If your query is urgent we can be contacted by telephone on *0871 22 00 260 24 hours a day. *For customers outside UK please call us on +44 115 912 9000 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Costin Manolache wrote: Httpd already has some support for that - .htaccess for example The trouble with .htaccess is that it only applies for data on the local filesystem. Url space created by other content handlers (such as proxy or ajp) is not covered. You're assuming that the tomcat admin has access to this config file, which is often not going to be the case. If you have to update a file on the httpd server, it's virtually no different to updating a file on the httpd server and issuing a graceful restart, and we already have that now. Well, this is an excelent point that I forgot to make :-) Yes, one of the most important benefits of having this dynamic config is that you _don't_ need direct access and editing of the httpd.conf. If the data about the cluster is by default stored in a file ( and without the Include tricks ), then it becomes possible to have the changes made using some program - either as a result of negotiation and tomcat JMX pushing new settings, or using config scripts, etc. You keep coming back to stored in a file. This file is stored where? On the httpd server? On the tomcat server? If it's on the httpd server, then you're back to editing a file on the webserver, and here there is no point - if you're already on the httpd server editing files, then you already probably have the power to issue a graceful restart anyway, which reloads the httpd service gracefully without breaking connections or otherwise inconveniencing users. If it's on the tomcat server, then how does httpd access the file? In band using either http or ajp? Out of band using SMB or NFS? Then there is the issue of securing that file - obviously you don't want just anybody accessing it or changing it. And then what if the httpd server is an appliance such as an SSL accelerator? No chance for a file there. That's a good solution. It assumes tomcat handles all configuration. It also assumes the master tomcat instances ( those that apache knows about when it starts up ) are up most of the time, so apache will allways be able to bootstrap. This is easily solved by specifying multiple possible bootstrap servers in the httpd.conf file, and then making sure that at least one of those servers is available. Big sites might even have a dedicated backend non-load-bearing tomcat server for this very purpose. Regardless of how the config info is communicated - negotiation, out of band, etc - it does help to have the cluster info outside of httpd.conf, in an easy to parse and generate file, and keep it updated. 100% true - which is why that info should probably be best kept within tomcat itself, and queried by httpd. The tomcat server.xml file already contains a lot of info about load balancing. Duplicating it in a new external file just adds new complications that could most probably be solved using what we already have. The idea of the static config in httpd.conf is firmly entrenched, and will take a while to dislodge. If we minimise the config in httpd, and put relevant dynamic config inside server.xml and/or the tomcat management app instead (and have httpd autodetect the status as it goes) we would have a really powerful dynamic system. Ok, I can live with that :-). httpd.conf is great because so many people are familiar with it. But in those google days, you can have 1000s of servers in a cluster. I don't know how this bootstraping mechanism can scale and if it is flexible enough. Why wouldn't it be? Remember a cluster and an httpd frontend will be connected to each other via a high speed network (typically). If tomcat says I am part of a cluster, here have the DNS names of 1000 of my closest friends in the cluster I see no reason why httpd would not be able to handle it gracefully. Again - it can't hurt too much to keep cluster info in a separate simple file. It opens the way to a lot of scripting, tool support, config pushes, etc. It adds the question of where this file lives, who will control access to it, etc. It doesn't sound like an elegant solution to me at all. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic updates and Apache v2.0
Graham Leggett wrote: Costin Manolache wrote: Httpd already has some support for that - .htaccess for example The trouble with .htaccess is that it only applies for data on the local filesystem. Url space created by other content handlers (such as proxy or ajp) is not covered. You're assuming that the tomcat admin has access to this config file, which is often not going to be the case. If you have to update a file on the httpd server, it's virtually no different to updating a file on the httpd server and issuing a graceful restart, and we already have that now. Well, this is an excelent point that I forgot to make :-) Yes, one of the most important benefits of having this dynamic config is that you _don't_ need direct access and editing of the httpd.conf. If the data about the cluster is by default stored in a file ( and without the Include tricks ), then it becomes possible to have the changes made using some program - either as a result of negotiation and tomcat JMX pushing new settings, or using config scripts, etc. You keep coming back to stored in a file. This file is stored where? On the httpd server? On the tomcat server? If it's on the httpd server, then you're back to editing a file on the webserver, and here there is no point - if you're already on the httpd server editing files, then you already probably have the power to issue a graceful restart anyway, which reloads the httpd service gracefully without breaking connections or otherwise inconveniencing users. If it's on the tomcat server, then how does httpd access the file? In band using either http or ajp? Out of band using SMB or NFS? Then there is the issue of securing that file - obviously you don't want just anybody accessing it or changing it. And then what if the httpd server is an appliance such as an SSL accelerator? No chance for a file there. Well using an URL could be a good idea, since it could be : - a static file, edited by admin, on the web-server or another web-server/tomcat - a dynamically generated file, PHP/JSP/Servlet/PERL, whatever. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Henri Gomez wrote: Well using an URL could be a good idea, since it could be : - a static file, edited by admin, on the web-server or another web-server/tomcat - a dynamically generated file, PHP/JSP/Servlet/PERL, whatever. An URL is a logical solution, which leaves us with two choices: - A specific out-of-band URL predefined in httpd where httpd is told get your config regularly here. This URL could be served by tomcat, or any other server. - In-band hop-by-hop header information that is deployed with responses from the backend servers. This has the advantage of having httpd learn about the status of backends on an up to the minute basis on each hit. httpd will already use certain in-band info, like whether a request was successful or not, or how long the request took. I prefer the in-band approach, as it's consistent. As has been pointed out, BEA Weblogic does this with success, so admins out there will not find this a foreign concept. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
RE: Dynamic updates and Apache v2.0
Graham Leggett wrote: Henri Gomez wrote: Well using an URL could be a good idea, since it could be : - a static file, edited by admin, on the web-server or another web-server/tomcat - a dynamically generated file, PHP/JSP/Servlet/PERL, whatever. An URL is a logical solution, which leaves us with two choices: - A specific out-of-band URL predefined in httpd where httpd is told get your config regularly here. This URL could be served by tomcat, or any other server. - In-band hop-by-hop header information that is deployed with responses from the backend servers. This has the advantage of having httpd learn about the status of backends on an up to the minute basis on each hit. httpd will already use certain in-band info, like whether a request was successful or not, or how long the request took. I prefer the in-band approach, as it's consistent. As has been pointed out, BEA Weblogic does this with success, so admins out there will not find this a foreign concept. +1 for X-Headers. Any other solution is IMO either too complicated at the moment or inadequate. We are already parsing headers from the backend so the information is inherently already there. All that is needed is to reed it, and update configuration accordingly. MT. smime.p7s Description: S/MIME cryptographic signature
Re: Dynamic updates and Apache v2.0
Graham Leggett wrote: Costin Manolache wrote: Httpd already has some support for that - .htaccess for example The trouble with .htaccess is that it only applies for data on the local filesystem. Url space created by other content handlers (such as proxy or ajp) is not covered. I know - it was just an example, showing that apache is not completely statically configured, and there are use cases they considered important enough to support. Unfortunately this doesn't extend to url space. You keep coming back to stored in a file. This file is stored where? On the httpd server? On the tomcat server? on httpd server. Right now the proposal ( or what I've seen ) is to define the workers in httpd.conf. I'm suggesting to have a separate file for workers, similar with htpasswd. If it's on the httpd server, then you're back to editing a file on the webserver, and here there is no point - if you're already on the httpd server editing files, then you already probably have the power to issue a graceful restart anyway, which reloads the httpd service gracefully without breaking connections or otherwise inconveniencing users. Again, if you have 100s load-balanced servers - and multithreaded apache servers - gracefull restart is not a convenient option. Just like you don't do a gracefull restart when you add a user in htpasswd or change a .htaccess file. Then there is the issue of securing that file - obviously you don't want just anybody accessing it or changing it. Same as htpasswd :-). And then what if the httpd server is an appliance such as an SSL accelerator? No chance for a file there. Well, that's an interesting case - so no httpd.conf either. Most appliances do have some file system emulation BTW. That's a good solution. It assumes tomcat handles all configuration. It also assumes the master tomcat instances ( those that apache knows about when it starts up ) are up most of the time, so apache will allways be able to bootstrap. This is easily solved by specifying multiple possible bootstrap servers in the httpd.conf file, and then making sure that at least one of those servers is available. Big sites might even have a dedicated backend non-load-bearing tomcat server for this very purpose. I'm not sure I understand what you want to prove. What is the problem with having the workers defined in a separate file ? Supporting the common case where no file system is available ? Yes, you can keep a dedicated tomcat server to maintain the list and pass it to apache. It will still be in some file, and it will still need to be dynamically updated and maintained - you just multiply the points of failure ( but move the hard problems to someone else :). It is true - it has the benefit of keeping apache side dumb and moving all logic to java - which actually makes sense. Regardless of how the config info is communicated - negotiation, out of band, etc - it does help to have the cluster info outside of httpd.conf, in an easy to parse and generate file, and keep it updated. 100% true - which is why that info should probably be best kept within tomcat itself, and queried by httpd. You're probably right. I'm starting to see your point. The idea of the static config in httpd.conf is firmly entrenched, and will take a while to dislodge. If we minimise the config in httpd, and put relevant dynamic config inside server.xml and/or the tomcat management app instead (and have httpd autodetect the status as it goes) we would have a really powerful dynamic system. Ok, I can live with that :-). httpd.conf is great because so many people are familiar with it. But in those google days, you can have 1000s of servers in a cluster. I don't know how this bootstraping mechanism can scale and if it is flexible enough. Why wouldn't it be? Remember a cluster and an httpd frontend will be connected to each other via a high speed network (typically). If tomcat says I am part of a cluster, here have the DNS names of 1000 of my closest friends in the cluster I see no reason why httpd would not be able to handle it gracefully. So basically you want to use tomcat as a config server for apache - all cluster config handled on java side, and apache just pulls the list of workers and updates from the config server instead of having it in it's own config. Fine - as long as we're not storing the cluster info in httpd.conf itself. I still think it may be easier to have it in a htpasswd-like file instead of getting it from a config server ( well, it is possible to use LDAP servers or similar stuff for config - the solution is proven, but it usually is more complicated than just a plain file ). To wrap this up - my concern was mosly about storing the cluster info in httpd.conf directives. If people feel having it in a separate file is more difficult than getting it from a tomcat config server - it's fine. Again - it can't hurt too much to keep cluster info in a separate simple file. It opens the
Re: Dynamic updates and Apache v2.0
Filip Hanik (lists) wrote: why are we so focused on dynamic this dynamic that, This thread is focused on the dynamic features, and was split out from the thread on the new work on ajp. Whether we do the dynamic features now or later isn't important, what is important is that any discussion happens separately from the ajp one. Please don't mix the issues back together again, otherwise we just end up with more confusion :( Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic updates and Apache v2.0
Graham Leggett wrote: Filip Hanik (lists) wrote: why are we so focused on dynamic this dynamic that, This thread is focused on the dynamic features, and was split out from the thread on the new work on ajp. Whether we do the dynamic features now or later isn't important, what is important is that any discussion happens separately from the ajp one. Please don't mix the issues back together again, otherwise we just end up with more confusion :( Well, you may keep the discussion separated, but the code will eventually be mixed. The reason I want to focus on dynamic cluster is that it does have a direct impact on the config format and code. If you look at tomcat5, jboss, etc - for dynamic configuration you need to design the software in a certain way. It's ok if the first versions will not have this feature, but every time you use pull to get config data, or set up some data structure - you need to think a bit how it will work with dynamic updates. One example - it would be better to use a syntax where the actual list of hosts in a cluster is stored in a separate file, that can be rewritten independently of httpd.conf ( like htpasswd ). If we don't do that - we'll either have to regenerate httpd.conf, or to add this later ( and confuse people with 2 ways to do something ), or have a lot of pain implementing it. If you can sometimes rewrite the code - the config format is very hard to change after people get used to. A lot in httpd.conf is from ncsa days. So for the stuff we add to mod_proxy - I think it is very important to think twice about where we want to go. Costin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Costin Manolache wrote: Well, you may keep the discussion separated, but the code will eventually be mixed. Eventually, but keeping each thing logically separate makes it easy to see where one feature might impact another. The devil with this is in the details, and I don't want to see important points getting lost amidst the volume. The reason I want to focus on dynamic cluster is that it does have a direct impact on the config format and code. But only after you've chosen how this dynamic config is going to work. Right now httpd does not have a concept of dynamic config, and although adding such a capability would be nice in the future it's a big change with a (likely) long timeframe. I don't think people want to wait that long :) So the alternative in the mean time is to make the communication between httpd and tomcat dynamic, so as to get the httpd people used to the idea that dynamic config is a good thing. :) If you look at tomcat5, jboss, etc - for dynamic configuration you need to design the software in a certain way. It's ok if the first versions will not have this feature, but every time you use pull to get config data, or set up some data structure - you need to think a bit how it will work with dynamic updates. One example - it would be better to use a syntax where the actual list of hosts in a cluster is stored in a separate file, that can be rewritten independently of httpd.conf ( like htpasswd ). You're assuming that the tomcat admin has access to this config file, which is often not going to be the case. If you have to update a file on the httpd server, it's virtually no different to updating a file on the httpd server and issuing a graceful restart, and we already have that now. What dynamic updates need to do is to auto-learn about the environment that it is in. And to do this, there is no better place to find out the info about a server cluster than from a member of that server cluster. Ideally tomcat should be able to pass to httpd info like hey, there is a new server in the pool, and it's called foo or do me a favour, I'm being taken out of the pool, so don't send me any new requests. Tomcat might add special hop-by-hop X- headers, or the OPTIONS idea might work. httpd would then update the scoreboard with info about the current pool of servers dynamically. Config of httpd would be as simple as ProxyPass /myWebapp ajp://seedserver/myWebapp The server seedserver could then tell httpd about all the other servers in the cluster dynamically. seedserver could resolve to one machine, or more than one machine. If you can sometimes rewrite the code - the config format is very hard to change after people get used to. A lot in httpd.conf is from ncsa days. So for the stuff we add to mod_proxy - I think it is very important to think twice about where we want to go. The idea of the static config in httpd.conf is firmly entrenched, and will take a while to dislodge. If we minimise the config in httpd, and put relevant dynamic config inside server.xml and/or the tomcat management app instead (and have httpd autodetect the status as it goes) we would have a really powerful dynamic system. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
RE: Dynamic updates and Apache v2.0
Graham Leggett wrote: Costin Manolache wrote: So the alternative in the mean time is to make the communication between httpd and tomcat dynamic, so as to get the httpd people used to the idea that dynamic config is a good thing. :) True, we will for start need only tree dynamic scenarios: A. Backend reports I'm to busy, get a life doode B. Backend reports I'm going down, no more new sessions please The third is from admin: C. Add and extra node to this balancer, or remove existing. Only the third case needs dynamic admin interface. The first two can come either from X-headers or AJP14. The easiest would be to add an extra handler to the lb. So: Location /x SetHandler proxy-balancer-status ...Security stuff... /Location This can even be a separate module using some APR_OPTIONAL_FN exposed by proxy_balancer, giving a simple html form to admins, probably even changing balancer factors etc... (first two cases). Since the module will use api exposed by the balancer any later implementation in httpd core will be possible. MT. smime.p7s Description: S/MIME cryptographic signature
Re: Dynamic updates and Apache v2.0
Graham Leggett wrote: Ideally tomcat should be able to pass to httpd info like hey, there is a new server in the pool, and it's called foo or do me a favour, I'm being taken out of the pool, so don't send me any new requests. Config of httpd would be as simple as ProxyPass /myWebapp ajp://seedserver/myWebapp The server seedserver could then tell httpd about all the other servers in the cluster dynamically. seedserver could resolve to one machine, or more than one machine. From what I experienced I can give an analogy with BEA WebLogic: The cluster client uses a URL to contact the WLS cluster. The URL has the form protocol://server1,server2,...:port The URL supports a list of servers to avoid bootstrapping problems in case of a single server breakdown. Usually two independent nodes of the cluster are statically configured in this URL. While the client starts, it tries to contact all servers configured in the URL. The first server that answers the request replies with a list of all cluster members. Then the client connects to all cluster members. The communication of the client with the cluster members includes several obvious ways of dynamically adding or removing cluster nodes: - a heartbeat protocol, so that the client detects a defect server by missing heartbeat packets - a configurable time interval after which the client updates its cluster server list by asking one of the already known cluster servers - a clean way of noticing the client by a cluster server that is in the process of being shut down To ensure consistency, all cluster members exchange membership data, so that they have the same view of the cluster topology. Furthermore the client loads communication code from the server. The code contains important information about load balancing. So the client learns from the cluster servers, if they should be used in a round-robin, or in a weight based way, and what their weights are. Again the cluster servers share this information via replication. This setup is working well in many real world scenarios. Maybe it indicates, that it is a good idea to learn at least cluster topology and load balancing configuration from statically configured bootstrap cluster nodes. Of course some information still needs to be managed dynamically inside the cluster. But with all the JMX support in Tomcat, that seems to be a place, where that should be feasible. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Graham Leggett wrote: The reason I want to focus on dynamic cluster is that it does have a direct impact on the config format and code. But only after you've chosen how this dynamic config is going to work. Right now httpd does not have a concept of dynamic config, and although adding such a capability would be nice in the future it's a big change with a (likely) long timeframe. I don't think people want to wait that long :) So the alternative in the mean time is to make the communication between httpd and tomcat dynamic, so as to get the httpd people used to the idea that dynamic config is a good thing. :) Httpd already has some support for that - .htaccess for example - and there are quite a few efforts to provide nice UI for apache config. That's the other side of dynamic configuration - it's not only about large cluster support, but also about beeing friendly to automatic config tools. Programatically editing httpd.conf ( with all the Includes and tricks ) is not trivial. One example - it would be better to use a syntax where the actual list of hosts in a cluster is stored in a separate file, that can be rewritten independently of httpd.conf ( like htpasswd ). You're assuming that the tomcat admin has access to this config file, which is often not going to be the case. If you have to update a file on the httpd server, it's virtually no different to updating a file on the httpd server and issuing a graceful restart, and we already have that now. Well, this is an excelent point that I forgot to make :-) Yes, one of the most important benefits of having this dynamic config is that you _don't_ need direct access and editing of the httpd.conf. If the data about the cluster is by default stored in a file ( and without the Include tricks ), then it becomes possible to have the changes made using some program - either as a result of negotiation and tomcat JMX pushing new settings, or using config scripts, etc. What dynamic updates need to do is to auto-learn about the environment that it is in. And to do this, there is no better place to find out the info about a server cluster than from a member of that server cluster. Ideally tomcat should be able to pass to httpd info like hey, there is a new server in the pool, and it's called foo or do me a favour, I'm being taken out of the pool, so don't send me any new requests. That's a good solution. It assumes tomcat handles all configuration. It also assumes the master tomcat instances ( those that apache knows about when it starts up ) are up most of the time, so apache will allways be able to bootstrap. Regardless of how the config info is communicated - negotiation, out of band, etc - it does help to have the cluster info outside of httpd.conf, in an easy to parse and generate file, and keep it updated. If you can sometimes rewrite the code - the config format is very hard to change after people get used to. A lot in httpd.conf is from ncsa days. So for the stuff we add to mod_proxy - I think it is very important to think twice about where we want to go. The idea of the static config in httpd.conf is firmly entrenched, and will take a while to dislodge. If we minimise the config in httpd, and put relevant dynamic config inside server.xml and/or the tomcat management app instead (and have httpd autodetect the status as it goes) we would have a really powerful dynamic system. Ok, I can live with that :-). httpd.conf is great because so many people are familiar with it. But in those google days, you can have 1000s of servers in a cluster. I don't know how this bootstraping mechanism can scale and if it is flexible enough. Again - it can't hurt too much to keep cluster info in a separate simple file. It opens the way to a lot of scripting, tool support, config pushes, etc. BTW - even tomcat, with all the JMX, still has some problems supporting persistence for the dynamic changes. I know Remy made a lot of improvements on saving server.xml, but I don't think it is as smooth as it can be. And IMO part of the problem is the config file format ( and all the features and complexity that are part of it). Costin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Dynamic updates and Apache v2.0
Hi all, I am starting a new thread for this, as it seems to be an important killer-app feature for any httpd v2.0 integration. People have said the config should be dynamically configurable - which part of the config should be dynamically configurable? In other words, would any of these senarios make sense: - A potential pool of servers (IP address/port combinations) is defined, and httpd gracefully handles the case where a server is missing. Adding a new server is as simple as starting it up on one of the predefined ip/ports combinations. An admin might configure an entire subnet as tomcat servers, and then add and remove backend servers at will. Easy to do, but a bit limiting. - httpd is informed via a special URL of updates to the servers on the backend, a bit like the management app in tomcat. This functionality might grow to being available to the whole httpd config. Being a URL, it would be pretected by standard mechanisms like SSL and basic authentication. Powerful, but a big change that will likely only appear in httpd v2.1 or v2.2. - httpd polls the backend servers using the OPTIONS method (as was recommended recently). In the info returned, httpd learns of the intended weighting of the servers, whether a server needs to be removed gracefully from the pool, or whether other servers have been added to the pool and should be included in the config. Here httpd needs only to be told about just one backend server, which will seed httpd with info about all the other servers. Out of the above three the third option seems to make the most sense. It's not very obtrusive, it can be used with backends other than tomcat. Actually updating the server list and weightings can be done via the tomcat management app, which already exists now (as opposed to a theoritical management app for httpd). Thoughts? Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Dynamic updates and Apache v2.0
First, by dynamic updates I mean changes to apache2 config that don't require a restart. For example, .htaccess files provide such a thing ( for a different area ). What updates ? There are several forms: 1. - add a new worker to a pool ( for example - expect big load, you buy more hardware, etc ). - gracefully remove a worker ( for upgrade for example ) - the implication is that sticky sessions will still go, no new sessions. - change parameters of a worker ( like balancing factors ). That may include advanced balancing. Note that for all this you don't actually need too much change on apache/mod_proxy - as long as you treat the pool as one entity, and maybe use a separate config file for the list of workers ( so you don't have to regenerate httpd.conf - which is quite difficult ). There are many ways to implement the communication between httpd processes and between apache and tomcat. 2. - add more webapps and virtual servers - for example in the case of an ISP. - reload webapps - that means some mappings and auth changes, in the case of tight integration you may want to have apache handle auth and mapping Costin Graham Leggett wrote: Hi all, I am starting a new thread for this, as it seems to be an important killer-app feature for any httpd v2.0 integration. People have said the config should be dynamically configurable - which part of the config should be dynamically configurable? In other words, would any of these senarios make sense: - A potential pool of servers (IP address/port combinations) is defined, and httpd gracefully handles the case where a server is missing. Adding a new server is as simple as starting it up on one of the predefined ip/ports combinations. An admin might configure an entire subnet as tomcat servers, and then add and remove backend servers at will. Easy to do, but a bit limiting. - httpd is informed via a special URL of updates to the servers on the backend, a bit like the management app in tomcat. This functionality might grow to being available to the whole httpd config. Being a URL, it would be pretected by standard mechanisms like SSL and basic authentication. Powerful, but a big change that will likely only appear in httpd v2.1 or v2.2. - httpd polls the backend servers using the OPTIONS method (as was recommended recently). In the info returned, httpd learns of the intended weighting of the servers, whether a server needs to be removed gracefully from the pool, or whether other servers have been added to the pool and should be included in the config. Here httpd needs only to be told about just one backend server, which will seed httpd with info about all the other servers. Out of the above three the third option seems to make the most sense. It's not very obtrusive, it can be used with backends other than tomcat. Actually updating the server list and weightings can be done via the tomcat management app, which already exists now (as opposed to a theoritical management app for httpd). Thoughts? Regards, Graham -- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dynamic updates and Apache v2.0
Costin Manolache wrote: 1. - add a new worker to a pool ( for example - expect big load, you buy more hardware, etc ). - gracefully remove a worker ( for upgrade for example ) - the implication is that sticky sessions will still go, no new sessions. - change parameters of a worker ( like balancing factors ). That may include advanced balancing. Note that for all this you don't actually need too much change on apache/mod_proxy - as long as you treat the pool as one entity, and maybe use a separate config file for the list of workers ( so you don't have to regenerate httpd.conf - which is quite difficult ). There are many ways to implement the communication between httpd processes and between apache and tomcat. Don't forget that in many cases Apache will be running on a machine entirely outside of your direct control - a good example would be a network appliance. There are many options for httpd to connect to the backend, and for the backend to connect to httpd, but there are limitations imposed by the environment where httpd will be running. Reading a file on disk is one example (you mentioned .htaccess files) - changing the file on disk does not mean httpd is necessarily going to pick up the changes straight away, and this assumes the admin has disk access at all. Whatever solution we start with needs to keep these limitations in mind. 2. - add more webapps and virtual servers - for example in the case of an ISP. - reload webapps - that means some mappings and auth changes, in the case of tight integration you may want to have apache handle auth and mapping Adding virtual servers requires an httpd config reload, this moves into the territory of having an httpd management app along the lines of the tomcat management app. Not a bad idea in itself, but a big project for httpd. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
RE: Dynamic updates and Apache v2.0
why are we so focused on dynamic this dynamic that, there is nothing about mod_proxy that is dynamic, and instead of delaying the stability and release of a working mod_proxy with a load balancer, make it work, make it work well, then add fancy features. mod_jk2 became way to cluttered this exact way, and never got to the point of workability to the point where any user could configure it just my $0.02 Filip -Original Message- From: news [mailto:[EMAIL PROTECTED] Behalf Of Costin Manolache Sent: Monday, July 26, 2004 11:30 AM To: [EMAIL PROTECTED] Subject: Re: Dynamic updates and Apache v2.0 First, by dynamic updates I mean changes to apache2 config that don't require a restart. For example, .htaccess files provide such a thing ( for a different area ). What updates ? There are several forms: 1. - add a new worker to a pool ( for example - expect big load, you buy more hardware, etc ). - gracefully remove a worker ( for upgrade for example ) - the implication is that sticky sessions will still go, no new sessions. - change parameters of a worker ( like balancing factors ). That may include advanced balancing. Note that for all this you don't actually need too much change on apache/mod_proxy - as long as you treat the pool as one entity, and maybe use a separate config file for the list of workers ( so you don't have to regenerate httpd.conf - which is quite difficult ). There are many ways to implement the communication between httpd processes and between apache and tomcat. 2. - add more webapps and virtual servers - for example in the case of an ISP. - reload webapps - that means some mappings and auth changes, in the case of tight integration you may want to have apache handle auth and mapping Costin Graham Leggett wrote: Hi all, I am starting a new thread for this, as it seems to be an important killer-app feature for any httpd v2.0 integration. People have said the config should be dynamically configurable - which part of the config should be dynamically configurable? In other words, would any of these senarios make sense: - A potential pool of servers (IP address/port combinations) is defined, and httpd gracefully handles the case where a server is missing. Adding a new server is as simple as starting it up on one of the predefined ip/ports combinations. An admin might configure an entire subnet as tomcat servers, and then add and remove backend servers at will. Easy to do, but a bit limiting. - httpd is informed via a special URL of updates to the servers on the backend, a bit like the management app in tomcat. This functionality might grow to being available to the whole httpd config. Being a URL, it would be pretected by standard mechanisms like SSL and basic authentication. Powerful, but a big change that will likely only appear in httpd v2.1 or v2.2. - httpd polls the backend servers using the OPTIONS method (as was recommended recently). In the info returned, httpd learns of the intended weighting of the servers, whether a server needs to be removed gracefully from the pool, or whether other servers have been added to the pool and should be included in the config. Here httpd needs only to be told about just one backend server, which will seed httpd with info about all the other servers. Out of the above three the third option seems to make the most sense. It's not very obtrusive, it can be used with backends other than tomcat. Actually updating the server list and weightings can be done via the tomcat management app, which already exists now (as opposed to a theoritical management app for httpd). Thoughts? Regards, Graham -- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --- Incoming mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.721 / Virus Database: 477 - Release Date: 7/16/2004 --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.721 / Virus Database: 477 - Release Date: 7/16/2004 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
