Dear Wiki user, You have subscribed to a wiki page or wiki category on "Httpd Wiki" for change notification.
The "PerformanceScalingOut" page has been changed by jmcg. The comment on this change is: wikifying Performance Scaling Out PDF, fixing some typos, converting to en_US. http://wiki.apache.org/httpd/PerformanceScalingOut?action=diff&rev1=1&rev2=2 -------------------------------------------------- See also: http://wiki.apache.org/httpd/PerformanceScalingUp + = Introduction = + + Building out a web server infrastructure is a large and multifaceted challenge. + The server infrastructure for any large web site is necessarily customized for + the needs and requirements of that site, thus it is very difficult to make valid + general statements about scaling technologies. This paper and its accompanying + ApacheCon presentation give a general overview of the field, touching upon + approaches and technologies rather than discussing them in depth. + + + == Why Would You Scale Out? == + + Scaling Out is a business decision. You may scale out because you cannot meet + your performance goals with a single web server. Alternatively, you may scale + out to meet reliability and uptime goals. There are many approaches to scaling + out, with varying price tags. So whatever your motivation, to scale out your + web infrastructure you will have to justify added expenses for server hardware, + network equipment, possibly software licenses and maintenance contracts, and + most certainly system administration time and resources. + + + = Building Out: Load Balancing = + + Scaling Out means adding more servers. The primary issue that arises when + servers are added is how to direct client transactions to multiple hosts. The user + does not know and does not need to know that multiple servers are in use. They + just want to point their browser to `www.example.com` and spend a lot of money + on your products or services. In this section we will review several techniques + to distribute client transactions across your hosts. + + + == Load Balancing with DNS == + + You have a great deal of control over where your users direct their transactions + by using the Domain Name Service (DNS) for your site. This seems obvious, + but it is critical to scaling. When your users connect to `www.example.com`, + they don’t care to which IP address this resolves. If you can manipulate this + resolution, you can send the user to whichever physical server you prefer, which + can be of great benefit to the infrastructure + + + === Distinct Servers for Distinct Services === + + One way to distribute transaction load across multiple physical servers is to give + each server a separate task. For your `www.example.com` site, use an + `images.example.com` server to serve static image content, a `secure.example.com` + server to handle SSL transactions, etc. This approach allows you to tune each server + for its specialized task. The downside is that this approach does not scale by + itself: once, for instance, your secure server runs out of processing headroom, + you will have to add more machines using one of the techniques described below. + + + === DNS Round-Robin === + If you operate multiple servers that perform identical functions, you can + distribute client transactions among them using Domain Name Server + Round-Robin. The principle behind this technique is that a single server hostname + resolves to a different IP address from your server pool for each DNS resolution + request. For instance, if you have three web servers with the IP addresses + + {{{ + 10.11.0.113 + 10.11.0.114 + 10.11.0.115 + }}} + + and you have your name server return each of those addresses in turn for + queries to your web server name (`www.scalingout.org`), roughly one third of + all clients will connect to each of your web servers. Since popular name server + implementations like bind implement this technique by default, it is very simple + to implement without any resource requirements besides control over your DNS zone. + + How "roughly" this works depends on many factors, over few of which you + have any control. Client-side resolvers cache query responses, as do intermediate + nameservers at ISPs and corporations. Large ISPs and corporations represent + many potential users, all of whom would be directed to the same web server for + as long as their nameserver caches the original lookup. However, across your + entire user population these discrepancies may even out. You can help this + process by reducing the cache timeout for query results in your zone file. An + example zone file that uses DNS Round-Robin is shown in Appendix A. + + DNS Round-Robin as a load balancing approach is often disparaged because + of its simplicity: it does not take into account the load on the servers, and + can not compensate for server outage. If a server goes down for any reason, + one third of all clients will still be directed to the nonfunctional server. If these + considerations are important to you, consider one of the more sophisticated load + balancing approaches described below. However, do not dismiss DNS Round- + Robin out of hand. Depending on your requirements, it may be all you need. + + + == Peer-based Load Balancing == + + You can turn a collection of individual servers into a cluster by using load + balancing techniques. In this section we will discuss Microsoft's approach. + + + === Windows Network Load Balancing === + + Windows Load Balancing Service (WLBS) technology has been available since + Windows NT Server 4.0, Enterprise Edition and is now included in Windows + Server 2003 under the name Network Load Balancing (NLB). Using Network + Load Balancing, you can turn up to 32 servers into a cluster. The service work + by having every machine assume the same IP address, and the same MAC address, + on the clustered interface(s). Incoming connections arrive at all members + of the cluster simultaneously from the network switch. The NLB software + communicates between cluster members over a unicast or multicast backchannel + and is responsible for the load balancing decisions. It sits between the network + card driver and the TCP/IP stack, and regulates which cluster member gets to + answer each incoming request. Cluster members whose NLB module doesn't + communicate with the other members get removed from the pool. This allows + NLB to provide High Availability as well as load balancing functionality. + + Because it operates below the TCP/IP layer, Network Load Balancing should + be compatible with any service that runs on the server machines. Each cluster + member has to be configured exactly the same. Please see your Windows Server + 2003 documentation for details. + + + == Load Balancing Appliance == + + The market for dedicated load balancing appliances is now quite crowded, with + offerings from vendors like Cisco, F5, Juniper, Foundry and many others vying + for your attention. These products can be pricey, but are powerful solutions for + load balancing your server farm. + + + === How a Load Balancer Works === + + Load balancing appliances or application switches sit between the web servers + and the outbound network connection and intelligently distribute traffic across + multiple web servers. They typically keep track of the load and availability of the + servers, and adjust their load balancing decisions accordingly. Many of these + products can operate on several layers of the network stack and can inspect + incoming requests to make load balancing decisions based on source address, + requested URI, cookies submitted etc. + + + === Linux Virtual Server === + + The Linux Virtual Server project is an open source load balancing and high + availability implementation. Its core module, IP Virtual Server, is included in + the kernel as of version 2.6.10. Auxiliary software like `ipvsadm` is only an install + away. If you are considering rolling your own load balancing solution, consider + Linux Virtual Server. + + The primary disadvantage of Linux Virtual Server is that it does not come + as a nice, shiny plug-and-play box with a support contract. Instead, it looks + more like an Erector Set (also known as ''Meccano'') of bits and pieces that + you get to integrate yourself. + However, this disadvantage can also be a strength: it allows you to build a + solution that best fits your needs. However, the absence of a 24x7 support plan + may upset your decision makers. You can find an example configuration for + Linux Virtual Server in Appendix B. + + + == Load Balancing with Tomcat == + The Tomcat application server and its `mod_jk` Apache HTTP Server connector + provide load balancing functionality that allows running a number of Tomcat + instances behind a single web server. + + This approach to Scaling Out is important for both performance and reliability + reasons. On the performance side, the web server can distribute resource + intensive application traffic among multiple application servers. From a reliability + point of view, it becomes possible to selectively take down Tomcat instances + for maintenance, without affecting overall availability of your application. Or, + you can periodically stop-start Tomcat instances to prevent the build up of is- + sues like memory leaks. Additionally, in the event a Tomcat instance crashes, + the load balancing `mod_jk` worker will automatically take it out of the pool until + it becomes available again. + + The load balancing functionality in `mod_jk` uses a round-robin algorithm to + assign requests to its workers. It maintains sticky sessions based on the Tomcat + session cookies, so requests that belong to the same application session will + always be routed to the same Tomcat instance. + A complete example of a Tomcat/`mod_jk` load balancing configuration is + available in Appendix C. You can also load balance a number of Tomcat servers + using the mod proxy balancer module that comes with Apache 2.2. This approach + is discussed in Section 2.5. + + + === Session Replication in a Tomcat Cluster === + + Sticky sessions allow for session persistence across a load balancer by having it + always route traffic that carries a specific session cookie to the same back-end + Tomcat server. However, when one of the back-end servers crashes or is taken + out of service for maintenance, the sessions it is serving at that moment are lost. + This is highly undesirable if these sessions represent customers' shopping carts + or banking transactions. Tomcat clustering and session replication prevents + session loss by copying session information from any cluster member to all the + other members as soon as a session is created. + + Clustering works over IP multicast. There are specific requirements for the + environment and session objects, which must be serializable. These require- + ments are discussed in the Tomcat documentation. An example configuration + is discussed in Appendix C. + + + == Load Balancing with Apache 2.2 == + + Version 2.2 of the Apache HTTP Server includes a load balancing proxy module, + mod proxy balancer. This add-on to mod proxy can balance incoming requests + between multiple back-end servers. It can persist connections to a particular + back-end based on a configurable Cookie key like `JSESSIONID` or `PHPSESSIONID`. + The configuration looks as follows: + + {{{ + Listen 80 + LogLevel debug + TransferLog logs/access_log + + LoadModule proxy_module modules/mod_proxy.so + LoadModule proxy_http_module modules/mod_proxy_http.so + LoadModule proxy_balancer_module modules/mod_proxy_balancer.so + + ProxyPass / balancer://mycluster/ + ProxyPassReverse / http://localhost:16180 + ProxyPassReverse / http://localhost:16280 + + <Proxy balancer://mycluster> + BalancerMember http://1.2.3.4:80 + BalancerMember http://1.2.3.5:80 + </Proxy> + }}} + + The configuration above will equally distribute requests between two back- + end web servers. If your back-end servers are running Tomcat, you can enable + sticky sessions as follows: + + {{{ + ... + <Proxy balancer://tccluster> + BalancerMember http://localhost:15180 route=tc1 + BalancerMember http://localhost:15280 route=tc2 + BalancerMember http://localhost:15380 route=tc3 + </Proxy> + + <Location /> + ProxyPass balancer://tccluster/ stickysession=JSESSIONID + Require all granted + </Location> + }}} + + The `server.xml` file of each Tomcat server has to be edited to include a + `jvmRoute` attribute in the Engine element. For instance, in the first Tomcat, + you have: + + {{{ + <!-- ... --> + <Server port="15105" shutdown="SHUTDOWN"> + <!-- ... --> + <Service name="Catalina"> + <Connector port="15180" maxHttpHeaderSize="8192" + maxThreads="150" minSpareThreads="25" maxSpareThreads="75" + enableLookups="false" redirectPort="8443" acceptCount="100" + connectionTimeout="20000" disableUploadTimeout="true" /> + <!-- ... --> + <Engine name="Catalina" defaultHost="localhost" jvmRoute="tc1"> + <!-- ... --> + </Engine> + </Service> + </Server> + }}} + + The `mod_proxy_balancer` module also contains a small management application, + which you can enable using the following configuration snippet: + + {{{ + <Location /balancer-manager> + SetHandler balancer-manager + # Your access control directives here + Order Allow,deny + Allow from all + # ... + </Location> + + The management feature requires the presence of `mod_status` in addition + to `mod_proxy_balancer`. As you can see, the load balancing reverse proxy in + Apache HTTP Server 2.2 is quite easy to set up. For more information about + the configuration options, please see the `mod_proxy` documentation. More + information on the algorithms for the load balancing decision can be found in + the mod proxy balancer documentation. + + + = Building Out: Separate Tiers = + + Most web applications can be separated into multiple distinct tiers: + # Web server tier(Apache, IIS, Sun ONE) + # Application server tier (Tomcat, PHP, WebLogic, etc.) + # Database server tier (MySQL, Oracle, Postgres, etc.) + Every tier has distinct and particular performance requirements. Moving + each tier to their own hardware allows you to tune and scale them individually. + The fact that all of the individual applications communicate with each other + over TCP/IP already makes this move even easier. + + + == The Web Server Tier == + + The '''Web Server''' tier communicates directly with the users. It is responsible for + maintaining connection with a wide variety of client browsers across potentially + slow and far-flung connections. This causes a markedly different load on the + operating system TCP stack than the long-lived, local, high speed connections + between web and application server, and between application server and the + database. The web tier can also be configured to serve the application's static + content: HTML pages, images, JavaScript, etc. It passes only the requests for + dynamically generated content (PHP scripts, JavaServer Pages, RSS feeds) on + to the application tier. The type of server used for this tier typically has one or + two CPUs and enough memory to fit the requisite number of httpd processes. + Storage is not a concern. + + + == The Application Server Tier == + + The '''Application Server''' tier generates all dynamic content. It receives requests + from the web tier and maintains connections to the database tier. The + operating system can be tuned specifically to run an application server platform + such as a Java virtual machine. The type of server used for this tier may + have multiple CPUs as required to run application threads. These servers have + more memory than the web servers as required by the application platform, but + storage is not important on this tier. + + + == The Database Server Tier == + The '''Database Server''' tier stores all application data. The application server + tier connects to the database tier using the JDBC protocol or native database + libraries. Database access can be a considerable bottleneck for application performance, + so performance is an important consideration for this tier. The type + of server used for this tier should have sufficient CPU power and RAM to run the + database application, and come with scalable, redundant storage like a RAID-5 + array. + + + = Designing Your Site for Scaling Out = + + + == Designing for a Load Balancer == + + A Load Balancer introduces an additional moving part to your web server infrastructure. + While most load balancer solutions do their best to appear transparent + to the application, you may find some issues that you can solve by properly + designing your application. + + The main issue arises with session persistence. The HTTP protocol is inherently + stateless, which is great for a load balancer: it can consider each incoming + request for itself and make a completely independent load balancing decision + based on its criteria. Session persistence potentially complicates this issue, especially + if a user's session exists only on the server that initially created it. If a + subsequent request from that user is directed to a different back-end server, the + session is lost. Most load balancing solutions solve this problem by consistently + directing requests from a particular IP address to the same back-end server. + Some can inspect incoming HTTP requests and make load balancing decisions + based on session cookies. + + These load balancer based fixes should be enough under most circumstances, + but your requirements may be more stringent: what if the user reconnects after + his lunch break and the load balancer has timed out because of inactivity from + the user's IP address? Or the user reconnects from a different IP address (let's + say she leaves one Starbucks and reconnects from the one across the street)? + Or the server that holds the user's session goes offline because of a crash or + maintenance? If it is important to you to maintain user sessions under circumstances + like these, you should build session persistence into your application. + Users sessions are likely to cause more reads than writes. You could write session + information to your back-end database, or use a special, fast database with + a write-through cache just for session maintenance. + + + = Conclusion = + Scaling out your web site is a mixed blessing. While you get to serve more + transactions and, presumably, do more business, the additional hardware, software + and network segments will also give you more intricacies to oversee. You + get to manage, maintain en secure a farm of servers instead of just one. The + configuration of your servers, and application software and content design will + be highly influenced by the infrastructure design decisions you make, and they + will be heavily intertwined. However, with judicious planning, scaling out can + be an efficient and effective solution to increased site demands. + + + = DNS Round-Robin Zone File = + + The following is a very basic DNS Zone file that uses Round-Robin DNS to + balance three web servers. + + {{{ + scalingout.org. 86400 IN SOA ns.scalingout.org. sctemme.scalingout.org. ( + 2006051401 ; Serial + 86400 ; refresh (1 day) + 7200 ; retry (2 hours) + 8640000 ; expire (10 days) + 86400 ) ; minimum (1 day) + + scalingout.org. IN NS bagheera.scalingout.org. + + gw IN A 10.11.0.1 + bagheera IN A 10.11.0.2 + + ; ... + + mail IN CNAME bagheera + ns IN CNAME bagheera + + + www IN A 10.11.0.113 + IN A 10.11.0.114 + IN A 10.11.0.115 + }}} + + + = Linux Virtual Server Configuration = + + This example uses a Linux Virtual Server director running `Ubuntu 5.10` (The + Breezy Badger). The outside interface of the Director has IP address `10.0.0.1`, + its inside interface is on `192.168.1.1`. Two back-end web servers are connected + to an internal interface of the Director. Their Ethernet interfaces are configured + for `192.168.1.2` and `192.168.1.3` respectively, and both have `192.168.1.1` for + default gateway. On the Director machine, the file `/etc/ipvsadm.rules` has the + following information: + + {{{ + # ipvsadm.rules + -A -t 10.0.0.1:80 -s rr + -a -t 10.0.0.1:80 -r 192.168.1.2:8080 -m -w 1 + -a -t 10.0.0.1:80 -r 192.168.1.3:8080 -m -w 1 + }}} + + and the file `/etc/defaults/ipvsadm` looks as follows: + + {{{ + # Do not edit! Use ’dpkg-reconfigure ipvsadm’. + AUTO="true" + DAEMON="none" + }}} + + The tool mentioned in the comment has interactive menus for the two variables. + This is all the configuration necessary to run Linux Virtual Server in + NAT mode: a reboot or the command `/etc/init.d/ipvsadm start` issued as + `root` starts the load balancer. + + + = Example Tomcat 5.5 Load Balancing Configuration = + + This example uses an Apache HTTP server with the `mod_jk` module, and three + Tomcat instances load balanced with sticky sessions. Note that the example + does not cover any access control or authorization. It is recommended that you + restrict access to the `mod_jk` status Worker. + + The following is an `httpd.conf` snippet that sets up `mod_jk`: + + {{{ + # Load the mod_jk module. You would obviously use the + # path to your own httpd installation. + LoadModule jk_module /Volumes/Files/asf/httpd-r415210w/modules/mod_jk.so + + # Mount your web applications. We are using the + # examples that come with Tomcat + JKMount /servlets-examples/* loadbalancer + JKMount /*.jsp loadbalancer + + # The mod_jk Manager application. Please implement appropriate + # access control on production systems. + JkMount /jkmanager/* jkstatus + + # Log mod_jk activity. You probably want a less verbose log level + JKLogFile logs/jk_log + JKLogLevel debug + + # The JKWorkerProperty directive is new as of mod_jk 1.2.7. It allows + # you to specify mod_jk worker configuration directives in httpd.conf + # instead of a separate workers.properties file + JKWorkerProperty worker.list=loadbalancer,jkstatus + + # Define three Tomcat instance workers + JKWorkerProperty worker.tc1.port=15109 + JKWorkerProperty worker.tc1.host=localhost + JKWorkerProperty worker.tc1.type=ajp13 + JKWorkerProperty worker.tc1.lbfactor=1 + + JKWorkerProperty worker.tc2.port=15209 + JKWorkerProperty worker.tc2.host=localhost + JKWorkerProperty worker.tc2.type=ajp13 + JKWorkerProperty worker.tc2.lbfactor=1 + + JKWorkerProperty worker.tc3.port=15309 + JKWorkerProperty worker.tc3.host=localhost + JKWorkerProperty worker.tc3.type=ajp13 + JKWorkerProperty worker.tc3.lbfactor=1 + + # Define a load balancer worker that uses the three + # Tomcat instances + JKWorkerProperty worker.loadbalancer.type=lb + JKWorkerProperty worker.loadbalancer.balance_workers=tc1, tc2, tc3 + + # Define the mod_jk status worker + JKWorkerProperty worker.jkstatus.type=status + }}} + + In the server.xml configuration file of each Tomcat, you need to set the + jvmRoute attribute of the Engine element to match the corresponding worker + name. For instance for the first Tomcat instance: + + {{{ + ... + <Engine name="Catalina" defaultHost="localhost" jvmRoute="tc1"> + ... + }}} + + If you don’t set this attribute, sticky sessions will not work! + + + == Session Replication in a Tomcat Cluster == + + The default `server.xml` configuration file that comes with Tomcat contains an + example clustering configuration. This example is commented out. To enable + clustering, find the `<Cluster> ... </Cluster>` element in the configuration + file and uncomment the entire element. If your test Tomcats are running on + the same host, change the `tcpListenPort` attribute of the Receiver element to a + unique value for every instance. For instance: + + {{{ + <Cluster className="org.apache.catalina.cluster.tcp.SimpleTcpCluster" + managerClassName="org.apache.catalina.cluster.session.DeltaManager" + expireSessionsOnShutdown="false" + useDirtyFlag="true" + notifyListenersOnReplication="true"> + + <!-- ... --> + + <Receiver + className="org.apache.catalina.cluster.tcp.ReplicationListener" + tcpListenAddress="auto" + tcpListenPort="15101" + + <!-- ... --> + }}} + + The second Tomcat instance gets `tcpListenPort="15201"`, the third `15301` + etc. Every web application that uses session replication has to have a distributable + element in its `web.xml` file. For instance, to make the Tomcat Servlet + examples distributable, edit `webapps/servlets-examples/WEB-INF/web.xml` for + every cluster member and add `<distributable />` to the `<web-app>` container. + + + == Testing your Deployment == + + To test the deployment as described above, place an `index.html` file in the + `webapps/ROOT` directory of every Tomcat instance with some distinctive text like + "This is Tomcat 1" (and 2, and 3 respectively). Subsequent requests from a + client to your load balancing Apache should show you the index pages from all + your back-end serves in succession. + + Testing sessions, sticky sessions and replication is not as simple. Browsers + very efficiently manage session cookies in a fashion completely transparent to the + user, so it is not easy to observe the exact behavior between client and server. + For this reason, it is often better to use a command-line client like `curl(1)`. For + instance, you can test session behavior using the following command: + + {{{ + $ curl -i -b /tmp/cookiejar -c /tmp/cookiejar \ + > http://localhost:15080/servlets-examples/servlet/SessionExample + }}} + + The first time you invoke this command, you will see a Set-Cookie: header + appear in the response, and the generated HTML page will display the session + ID. The last element of the session ID should be the `jvmRoute` value of the + Tomcat instance that received the request. Subsequent requests will show the + same session ID in the page, since requests will be routed to the same instance + using the sticky session. Store some data in the session by invoking: + + {{{ + $ curl -i -b /tmp/cookiejar -c /tmp/cookiejar \ + > http://localhost:15080/servlets-examples/servlet/SessionExample\ + > ?dataname=foo\&datavalue=bar + }}} + + The value ''foo = bar'' shoud now appear under The following data is in your + session in the generated HTML page. Shut the receiving Tomcat down by calling + its `bin/shutdown.sh` script, and try the request again. If you didn't configure + clustering, you will receive a response from a different Tomcat instance with a + new session ID. If you did configure clustering, however, the new Tomcat will + have replicated the session data from the first one and you should see the same + session ID, and the same key/value pair stored in the session. + --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
