Updating documenation and how to generate it.
Project: http://git-wip-us.apache.org/repos/asf/incubator-blur/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-blur/commit/bcb69372 Tree: http://git-wip-us.apache.org/repos/asf/incubator-blur/tree/bcb69372 Diff: http://git-wip-us.apache.org/repos/asf/incubator-blur/diff/bcb69372 Branch: refs/heads/blur-console-v2 Commit: bcb693729546ab2f4d5104933ec870570ff0605d Parents: 3453eb7 Author: Aaron McCurry <[email protected]> Authored: Thu Oct 31 15:03:44 2013 -0400 Committer: Aaron McCurry <[email protected]> Committed: Thu Oct 31 15:04:35 2013 -0400 ---------------------------------------------------------------------- docs/cluster-setup.html | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-blur/blob/bcb69372/docs/cluster-setup.html ---------------------------------------------------------------------- diff --git a/docs/cluster-setup.html b/docs/cluster-setup.html index c29a4ea..05949cd 100644 --- a/docs/cluster-setup.html +++ b/docs/cluster-setup.html @@ -123,8 +123,12 @@ blur.cluster.name=default # the table descriptor will be required for all tables. blur.cluster.default.table.uri=hdfs://namenode/blur/tables</code> -</pre> - </p> +</pre></p> + <h4>Default Properties</h4> + <table class="table-bordered table-striped table-condensed"> + <tr><td>Property</td><td>Description</td></tr> +<tr><td>blur.zookeeper.timeout (90000)</td><td>The zookeeper session timeout</td></tr><tr><td>blur.query.max.results.fetch (1000)</td><td>The maximum number of results that can be fetched in a single request</td></tr><tr><td>blur.query.max.row.fetch (100)</td><td>The maximum number of rows that can be fetched in a single request</td></tr><tr><td>blur.query.max.record.fetch (1000)</td><td>The maximum number of records that can be fetched in a single request</td></tr><tr><td>blur.metrics.reporters</td><td>Setup metric reporter</td></tr> + </table> <h3 id="general-hadoop">Hadoop</h3> <p> The current version of Blur has Hadoop 1.2.1 embedded in the "apache-blur-*/lib/hadoop-1.2.1" path. However if @@ -145,7 +149,7 @@ export HADOOP_HOME=<path to your Hadoop install directory></code> <p> These are the default settings for the shard server that can be overridden in the blur-site.properties file. Consider increasing the various thread pool counts (*.thread.count). The blur.controller.server.remote.thread.count is very important to increase for larger clusters, basically one thread is used per shard server per query. Some production clusters have set this thread pool to 2000 or more threads. </p> -<h4>Properties</h4> +<h4>Default Properties</h4> <table class="table-bordered table-striped table-condensed"> <tr><td>Property</td><td>Description</td></tr> <tr><td>blur.controller.hostname</td><td>Sets the hostname for the controller, if blank the hostname is automatically detected</td></tr><tr><td>blur.controller.bind.address (0.0.0.0)</td><td>The binding address of the controller</td></tr><tr><td>blur.controller.bind.port (40010)</td><td>The default binding port of the controller server</td></tr><tr><td>blur.controller.shard.connection.timeout (60000)</td><td>The connection timeout, NOTE: this will be the maximum amount of time you can wait for a query.</td></tr><tr><td>blur.controller.server.thrift.thread.count (32)</td><td>The number of threads used for thrift requests</td></tr><tr><td>blur.controller.server.remote.thread.count (64)</td><td>The number of threads used for remote thrift requests to the shards server. This should be a large number.</td></tr><tr><td>blur.controller.thrift.selector.threads (2)</td><td>The number of threads used for selector processing inside the thrift server.</td></tr><tr><td>blur.controller.thrift.ma x.read.buffer.bytes (9223372036854775807)</td><td>The maximum number of bytes used for reading requests in the thrift server.</td></tr><tr><td>blur.controller.thrift.accept.queue.size.per.thread (4)</td><td>The size of the blocking queue per selector thread for passing accepted connections to the selector thread.</td></tr><tr><td>blur.controller.remote.fetch.count (100)</td><td>The number of hits to fetch per request to the shard servers</td></tr><tr><td>blur.controller.retry.max.fetch.retries (3)</td><td>The max number of retries to the shard server when there is an error during fetch</td></tr><tr><td>blur.controller.retry.max.mutate.retries (3)</td><td>The max number of retries to the shard server when there is an error during mutate</td></tr><tr><td>blur.controller.retry.max.default.retries (3)</td><td>The max number of retries to the shard server when there is an error during all other request</td></tr><tr><td>blur.controller.retry.fetch.delay (500)</td><td>The starting backoff delay for the first retry for a fetch errors</td></tr><tr><td>blur.controller.retry.mutate.delay (500)</td><td>The starting backoff delay for the first retry for a mutate errors</td></tr><tr><td>blur.controller.retry.default.delay (500)</td><td>The starting backoff delay for the first retry for a all other request errors</td></tr><tr><td>blur.controller.retry.max.fetch.delay (2000)</td><td>The ending backoff delay for the last retry for a fetch errors</td></tr><tr><td>blur.controller.retry.max.mutate.delay (2000)</td><td>The ending backoff delay for the last retry for a mutate errors</td></tr><tr><td>blur.controller.retry.max.default.delay (2000)</td><td>The ending backoff delay for the last retry for a all other request errors</td></tr><tr><td>blur.gui.controller.port (40080)</td><td>The http status page port for the controller server</td></tr> @@ -192,7 +196,7 @@ Swap can kill java perform, you may want to consider disabling swap.</div> These are the default settings for the shard server that can be overridden in the blur-site.properties file. Consider increasing the various thread pool counts (*.thread.count). Also the blur.max.clause.count sets the BooleanQuery max clause count for Lucene queries. </p> - <h4>Properties</h4> + <h4>Default Properties</h4> <table class="table-bordered table-striped table-condensed"> <tr><td>Property</td><td>Description</td></tr> <tr><td>blur.shard.hostname</td><td>The hostname for the shard, if blank the hostname is automatically detected</td></tr><tr><td>blur.shard.bind.address (0.0.0.0)</td><td>The binding address of the shard</td></tr><tr><td>blur.shard.bind.port (40020)</td><td>The default binding port of the shard server</td></tr><tr><td>blur.shard.data.fetch.thread.count (8)</td><td>The number of fetcher threads</td></tr><tr><td>blur.shard.server.thrift.thread.count (8)</td><td>The number of the thrift threads</td></tr><tr><td>blur.shard.thrift.selector.threads (2)</td><td>The number of threads used for selector processing inside the thrift server.</td></tr><tr><td>blur.shard.thrift.max.read.buffer.bytes (9223372036854775807)</td><td>The maximum number of bytes used for reading requests in the thrift server.</td></tr><tr><td>blur.shard.thrift.accept.queue.size.per.thread (4)</td><td>The size of the blocking queue per selector thread for passing accepted connections to the selector thread.</td></tr><tr ><td>blur.shard.opener.thread.count (8)</td><td>The number of threads that are >used for opening >indexes</td></tr><tr><td>blur.shard.cache.max.querycache.elements >(128)</td><td>The number of cached >queries</td></tr><tr><td>blur.shard.cache.max.timetolive (60000)</td><td>The >time to live for the cache >query</td></tr><tr><td>blur.shard.filter.cache.class >(org.apache.blur.manager.DefaultBlurFilterCache)</td><td>Default >implementation of the blur cache filter, which is a pass through filter that >does nothing</td></tr><tr><td>blur.shard.index.warmup.class >(org.apache.blur.manager.indexserver.DefaultBlurIndexWarmup)</td><td>Default >Blur index warmup class that warms the fields provided in the table >descriptor</td></tr><tr><td>blur.shard.index.warmup.throttle >(30000000)</td><td>Throttles the warmup to 30MB/s across all the warmup >threads</td></tr><tr><td>blur.shard.block.cache.version (v2)</td><td>By >default the v2 version of the block cache is >enabled</td></tr><tr><td>blur.shard.block.cach e.total.size</td><td>By default the total amount of memory block cache will use is -XX:MaxDirectMemorySize - 64 MiB</td></tr><tr><td>blur.shard.blockcache.direct.memory.allocation (true)</td><td>v1 version of block cache only. By default the block cache using off heap memory</td></tr><tr><td>blur.shard.blockcache.slab.count (-1)</td><td>v1 version of block cache only. The slabs in the blockcache are automatically configured by default (-1) otherwise 1 slab equals 128MB. The auto config is detected through the MaxDirectoryMemorySize provided to the JVM</td></tr><tr><td>blur.shard.block.cache.v2.fileBufferSize (8192)</td><td>v2 version of block cache only. File buffer size, this is the buffer size used to read and write to data to HDFS. For production this will likely be increased.</td></tr><tr><td>blur.shard.block.cache.v2.cacheBlockSize (8192)</td><td>v2 version of block cache only. The is the size of the blocks in the off heap cache, it is good practice to have this match 'blur.s hard.block.cache.v2.fileBufferSize'. For production this will likely be increased.</td></tr><tr><td>blur.shard.block.cache.v2.cacheBlockSize.filter (33554432)</td><td>blur.shard.block.cache.v2.cacheBlockSize.<ext>=</td></tr><tr><td>blur.shard.block.cache.v2.store (OFF_HEAP)</td><td>v2 version of block cache only. This is used to control if the block are created on or off heap. Values are OFF_HEAP | ON_HEAP</td></tr><tr><td>blur.shard.block.cache.v2.read.cache.ext</td><td>v2 version of block cache only. This specifies what file types should be cached during reads. Comma delimited list.</td></tr><tr><td>blur.shard.block.cache.v2.read.nocache.ext (fdt)</td><td>v2 version of block cache only. This specifies what file types should NOT be cached during reads. Comma delimited list.</td></tr><tr><td>blur.shard.block.cache.v2.read.default (true)</td><td>v2 version of block cache only. This specifies the default behavior if a file type is not specified in the cache or nocache lists during reads. Values true | false</td></tr><tr><td>blur.shard.block.cache.v2.write.cache.ext</td><td>v2 version of block cache only. This specifies what file types should be cached during writes. Comma delimited list.</td></tr><tr><td>blur.shard.block.cache.v2.write.nocache.ext (fdt)</td><td>v2 version of block cache only. This specifies what file types should NOT be cached during writes. Comma delimited list.</td></tr><tr><td>blur.shard.block.cache.v2.write.default (true)</td><td>v2 version of block cache only. This specifies the default behavior if a file type is not specified in the cache or nocache lists during writes. Values true | false</td></tr><tr><td>blur.shard.buffercache.1024 (8192)</td><td>The number of 1K byte buffers</td></tr><tr><td>blur.shard.buffercache.8192 (8192)</td><td>The number of 8K byte buffers</td></tr><tr><td>blur.shard.safemodedelay (5000)</td><td>The number of milliseconds to wait for the cluster to settle once changes have ceased</td></tr><tr><td>blur.sha rd.time.between.commits (30000)</td><td>The default time between index commits</td></tr><tr><td>blur.shard.time.between.refreshs (3000)</td><td>The default time between index refreshs</td></tr><tr><td>blur.shard.merge.thread.count (3)</td><td>The max number of threads used during index merges</td></tr><tr><td>blur.max.clause.count (1024)</td><td>The maximum number of clauses in a BooleanQuery</td></tr><tr><td>blur.indexmanager.search.thread.count (8)</td><td>The number of thread used for parallel searching in the index manager</td></tr><tr><td>blur.indexmanager.mutate.thread.count (8)</td><td>The number of thread used for parallel mutating in the index manager</td></tr><tr><td>blur.shard.internal.search.thread.count (8)</td><td>The number of threads used for parallel searching in the index searchers</td></tr><tr><td>blur.shard.warmup.thread.count (8)</td><td>Number of threads used for warming up the index</td></tr><tr><td>blur.shard.fetchcount (100)</td><td>The fetch count per Lucen e search, this fetches pointers to hits</td></tr><tr><td>blur.max.heap.per.row.fetch (10000000)</td><td>Heap limit on row fetch, once this limit has been reached the request will return</td></tr><tr><td>blur.max.records.per.row.fetch.request (1000)</td><td>The maximum number of records in a single row fetch</td></tr><tr><td>blur.gui.shard.port (40090)</td><td>The http status page port for the shard server</td></tr>
