Adding some more docs to help deal with some gotchas.
Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/3eae6f88 Tree: http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/3eae6f88 Diff: http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/3eae6f88 Branch: refs/heads/master Commit: 3eae6f88b0f9f0b4b10c27385e862b647e190c82 Parents: 83bfb70 Author: Aaron McCurry <[email protected]> Authored: Wed Aug 28 15:44:13 2013 -0400 Committer: Aaron McCurry <[email protected]> Committed: Wed Aug 28 15:44:13 2013 -0400 ---------------------------------------------------------------------- docs/cluster-setup.html | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/3eae6f88/docs/cluster-setup.html ---------------------------------------------------------------------- diff --git a/docs/cluster-setup.html b/docs/cluster-setup.html index 8657acc..36452b5 100644 --- a/docs/cluster-setup.html +++ b/docs/cluster-setup.html @@ -216,6 +216,28 @@ export BLUR_NUMBER_OF_CONTROLLER_SERVER_INSTANCES_PER_MACHINE=1</code></pre> <div class="page-header"> <h1 id="shard">Shard Server Configuration</h1> </div> +<h3>Minimum Settings to Configure</h3> +<p> +It is highly recommended that the ulimits are increase on the server specifically: +<ul> +<li>open files</li> +<li>max user processes</li> +</ul> +<br/> +In Hadoop the dfs.datanode.max.xcievers should be increased to at least 4096 if not more. +<pre> +<code class="bash"><property> + <name>dfs.datanode.max.xcievers</name> + <value>4096</value> +</property></code></pre> +<br/> +In blur-env.sh set the cache memory for the shard processes. DO NOT over allocate this will +likely crash your server. +<pre><code class="bash">-XX:MaxDirectMemorySize=13g</code></pre> +<div class="bs-callout bs-callout-warning"><h4>Caution</h4> +Swap can kill java perform, you may want to consider disabling swap.</div> + +</p> <h3 id="shard-blur-site">blur-site.properties</h3> <p> These are the default settings for the shard server that can be overridden in the blur-site.properties file. Consider increasing the various thread pool counts (*.thread.count). Also the blur.max.clause.count sets the BooleanQuery max clause count for Lucene queries.
