Author: dmeil
Date: Wed Aug 24 21:05:38 2011
New Revision: 1161273
URL: http://svn.apache.org/viewvc?rev=1161273&view=rev
Log:
HBASE-4249 - performance.xml (adding network section)
Modified:
hbase/trunk/src/docbkx/performance.xml
Modified: hbase/trunk/src/docbkx/performance.xml
URL:
http://svn.apache.org/viewvc/hbase/trunk/src/docbkx/performance.xml?rev=1161273&r1=1161272&r2=1161273&view=diff
==============================================================================
--- hbase/trunk/src/docbkx/performance.xml (original)
+++ hbase/trunk/src/docbkx/performance.xml Wed Aug 24 21:05:38 2011
@@ -24,7 +24,59 @@
<para>Watch out for swapping. Set swappiness to 0.</para>
</section>
</section>
-
+ <section xml:id="perf.network">
+ <title>Network</title>
+ <para>
+ Perhaps the most important factor in avoiding network issues degrading
Hadoop and HBbase performance is the switching hardware
+ that is used, decisions made early in the scope of the project can cause
major problems when you double or triple the size of your cluster (or more).
+ </para>
+ <para>
+ Important items to consider:
+ <itemizedlist>
+ <listitem>Switching capacity of the device</listitem>
+ <listitem>Number of systems connected</listitem>
+ <listitem>Uplink capacity</listitem>
+ </itemizedlist>
+ </para>
+ <section xml:id="perf.network.1switch">
+ <title>Single Switch</title>
+ <para>The single most important factor in this configuration is that the
switching capacity of the hardware is capable of
+ handling the traffic which can be generated by all systems connected to
the switch. Some lower priced commodity hardware
+ can have a slower switching capacity than could be utilized by a full
switch.
+ </para>
+ </section>
+ <section xml:id="perf.network.2switch">
+ <title>Multiple Switches</title>
+ <para>Multiple switches are a potential pitfall in the architecture.
The most common configuration of lower priced hardware is a
+ simple 1Gbps uplink from one switch to another. This often overlooked
pinch point can easily become a bottleneck for cluster communication.
+ Especially with MapReduce jobs that are both reading and writing a lot
of data the communication across this uplink could be saturated.
+ </para>
+ <para>Mitigation of this issue is fairly simple and can be accomplished
in multiple ways:
+ <itemizedlist>
+ <listitem>Use appropriate hardware for the scale of the cluster which
you're attempting to build.</listitem>
+ <listitem>Use larger single switch configurations i.e. single 48 port
as opposed to 2x 24 port</listitem>
+ <listitem>Configure port trunking for uplinks to utilize multiple
interfaces to increase cross switch bandwidth.</listitem>
+ </itemizedlist>
+ </para>
+ </section>
+ <section xml:id="perf.network.multirack">
+ <title>Multiple Racks</title>
+ <para>Multiple rack configurations carry the same potential issues as
multiple switches, and can suffer performance degradation from two main areas:
+ <itemizedlist>
+ <listitem>Poor switch capacity performance</listitem>
+ <listitem>Insufficient uplink to another rack</listitem>
+ </itemizedlist>
+ If the the switches in your rack have appropriate switching capacity to
handle all the hosts at full speed, the next most likely issue will be caused
by homing
+ more of your cluster across racks. The easiest way to avoid issues when
spanning multiple racks is to use port trunking to create a bonded uplink to
other racks.
+ The downside of this method however, is in the overhead of ports that
could potentially be used. An example of this is, creating an 8Gbps port
channel from rack
+ A to rack B, using 8 of your 24 ports to communicate between racks gives
you a poor ROI, using too few however can mean you're not getting the most out
of your cluster.
+ </para>
+ <para>Using 10Gbe links between racks will greatly increase performance,
and assuming your switches support a 10Gbe uplink or allow for an expansion
card will allow you to
+ save your ports for machines as opposed to uplinks.
+ </para>
+
+ </section>
+ </section> <!-- network -->
<section xml:id="jvm">
<title>Java</title>
@@ -56,7 +108,7 @@
</section>
<section xml:id="perf.configurations">
- <title>Configurations</title>
+ <title>HBase Configurations</title>
<para>See <xref linkend="recommended_configurations" />.</para>
