Merge branch '1.5.2-SNAPSHOT' into 1.6.1-SNAPSHOT
Conflicts:
test/system/stress/README.md
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/2a17a7a2
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/2a17a7a2
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/2a17a7a2
Branch: refs/heads/1.6.1-SNAPSHOT
Commit: 2a17a7a2b4eebdfb0a89f51b97adaab52904bc95
Parents: 934ca6f e01a720
Author: Bill Havanki <bhava...@cloudera.com>
Authored: Thu Jun 26 17:29:00 2014 -0400
Committer: Bill Havanki <bhava...@cloudera.com>
Committed: Thu Jun 26 17:29:00 2014 -0400
----------------------------------------------------------------------
test/system/stress/README.md | 95 +++++++++++++++++++++++----
test/system/stress/reader.sh | 42 ++++++------
test/system/stress/readers | 17 +++++
test/system/stress/start-readers.sh | 40 +++++++++++
test/system/stress/start-writers.sh | 40 +++++++++++
test/system/stress/stop-readers.sh | 36 ++++++++++
test/system/stress/stop-writers.sh | 36 ++++++++++
test/system/stress/stress-env.sh.example | 56 ++++++++++++++++
test/system/stress/writer.sh | 44 ++++++-------
test/system/stress/writers | 17 +++++
10 files changed, 367 insertions(+), 56 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/2a17a7a2/test/system/stress/README.md
----------------------------------------------------------------------
diff --cc test/system/stress/README.md
index 9546b21,e400d4b..9eb9fdc
--- a/test/system/stress/README.md
+++ b/test/system/stress/README.md
@@@ -15,20 -15,86 +15,91 @@@ See the License for the specific langua
limitations under the License.
-->
-
Accumulo Stress Test
- --------------------
+ ====================
+
+ This is an attempt to observe the behavior Accumulo displays when compacting
and
+ reading cells. There are two components to this package:
+ `org.apache.accumulo.test.stress.random.Write` and
+ `org.apache.accumulo.test.stress.random.Scan`.
+
+ Since this test is meant to stress the system, there is no pre-defined test
+ period and no success criteria that we can programmatically check. During a
+ successful test period, the Accumulo cluster should remain stable and
+ functional. Possible failure conditions include component failures, such as
+ tablet servers running out of memory.
+
+ The `org.apache.accumulo.test.stress.random.Write` utility provides facilities
+ for writing random sized cells. Users can configure minimum and maximum sized
+ portions of a cell. The portions users can configure are the row, column
family,
+ column qualifier and value. Note that the sizes are uniformly distributed
+ between the minimum and maximum values.
+
+ The `org.apache.accumulo.test.stress.random.Scan` utility provides users with
+ the ability to query tables generated by the Write. It will pick a tablet at
+ random and scan the entire range. The amount of times this process is
performed
+ is user configurable. By default, it happens 1,024 times. Users can also
specify
+ whether or not the scan should be isolated.
+
+ There is no shared state intended by either of these services. This allows
+ multiple clients to be run in parallel, either on the same host or distributed
+ across hosts.
+
+ ## Prerequisites
+
+ The utilities are run on a normal, initialized Accumulo cluster. By default,
+ they only work with a table named "stress_test".
+
+ The start and stop scripts rely on pssh. Before running them, you will need
+ to install pssh on the machines that will be controlled by them.
+
+ ## Running
+
+ Copy `stress-env.sh.example` to `stress-env.sh` and edit it, supplying at
least
+ correct values for the USERPASS and INSTANCE variables. The remainder of this
+ file describes and provides defaults for many of the configuration options
+ for the stress utilities.
+
+ ### Individual Execution
+
+ Before starting a fresh write test, clear out the test table using the
Accumulo
+ shell.
+
+ > deletetable -f stress_test
+ > createtable stress_test
+
+ To run a writer:
+
+ $ ./writer.sh
+
+ The writer will begin logging into a "logs" subdirectory, using timestamped
+ filenames. You can stop the writer by killing it or using Control-C.
+
+ To run a reader:
+
+ $ ./reader.sh
+
+ The reader logs like the writer. By default, the reader reads forever, but
+ it can be configured for a fixed number of scan iterations.
- This is an attempt to observe the behavior Accumulo displays when compacting
and reading these cells. There are two components to this package:
`org.apache.accumulo.test.stress.random.Write` and
`org.apache.accumulo.test.stress.random.Scan`.
+ ### Group Execution
- Since this test is meant to stress the system, there is no pre-defined test
period and no success criteria that we can programmatically check. During a
successful test period, the Accumulo cluster should remain stable and
functional. Possible failure conditions include component failures, such as
tablet servers running out of memory.
+ To run writers across multiple machines, create a writers file and list each
+ host that will run a writer, one per line.
- The `org.apache.accumulo.test.stress.random.Write` utility provides
facilities for writing random sized cells. Users can configure minimum and
maximum sized portions of a cell. The portions users can configure are the row,
column family, column qualifier and value. Note that the sizes are uniformly
distributed between the minimum and maximum values
+ Finally, to start and stop the writers:
- The `org.apache.accumulo.test.stress.random.Scan` utility provides users with
the ability to query tables generated by the Write. It will pick a tablet at
random and scan the entire range. The amount of times this process is performed
is user configurable. By default, it happens 1,024 times. Users can also
specify whether or not the scan should be isolated or not.
+ $ ./start-writers.sh
+ $ ./stop-writers.sh
- There is no shared state intended by either of these services. This allows
multiple clients to be run in parallel, either on the same host or
distributedacross hosts.
+ For readers, make a similar readers file, and use `start-readers.sh` and
+ `stop-readers.sh`.
- See the provided `test/system/stress/writer.sh` and
`test/system/stress/reader.sh` scripts for example configuration options and
how to launch the processes.
+ The start scripts copy out stress-env.sh to each of the writers and readers,
+ overwriting what is there, so you only need to edit the file on the host from
+ which you run those scripts.
+
- Note that in versions of Accumulo greater than 1.6.0, users should verify
that the cell sizes they supply meet the criteria set forth by the key
constraints on their tables. By default, tables are configured to reject keys
that are are greater than 1MB.
++Note that in versions of Accumulo greater than 1.6.0, users should verify that
++the cell sizes they supply meet the criteria set forth by the key constraints
++on their tables. By default, tables are configured to reject keys that are
++greater than 1MB.
