ACCUMULO-2847 Add "practicaly" documentation to the user manual for replication.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/7894feda Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/7894feda Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/7894feda Branch: refs/heads/master Commit: 7894fedad0ea093e61dcdbb35e9df731ba85a971 Parents: f883102 Author: Josh Elser <[email protected]> Authored: Fri Jun 13 23:55:06 2014 -0400 Committer: Josh Elser <[email protected]> Committed: Fri Jun 13 23:55:06 2014 -0400 ---------------------------------------------------------------------- .../org/apache/accumulo/core/conf/Property.java | 2 +- docs/src/main/asciidoc/chapters/replication.txt | 109 +++++++++++++++++++ 2 files changed, 110 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/accumulo/blob/7894feda/core/src/main/java/org/apache/accumulo/core/conf/Property.java ---------------------------------------------------------------------- diff --git a/core/src/main/java/org/apache/accumulo/core/conf/Property.java b/core/src/main/java/org/apache/accumulo/core/conf/Property.java index 29993b3..e35a9c3 100644 --- a/core/src/main/java/org/apache/accumulo/core/conf/Property.java +++ b/core/src/main/java/org/apache/accumulo/core/conf/Property.java @@ -469,7 +469,7 @@ public enum Property { REPLICATION_MIN_THREADS("replication.receiver.min.threads", "1", PropertyType.COUNT, "Minimum number of threads for replication"), REPLICATION_THREADCHECK("replication.receiver.threadcheck.time", "30s", PropertyType.TIMEDURATION, "The time between adjustments of the replication thread pool."), REPLICATION_MAX_UNIT_SIZE("replication.max.unit.size", "64M", PropertyType.MEMORY, "Maximum size of data to send in a replication message"), - REPLICATION_WORK_ASSIGNER("replication.work.assigner", "org.apache.accumulo.master.replication.SequentialWorkAssigner", PropertyType.CLASSNAME, + REPLICATION_WORK_ASSIGNER("replication.work.assigner", "org.apache.accumulo.master.replication.UnorderedWorkAssigner", PropertyType.CLASSNAME, "Replication WorkAssigner implementation to use"), REPLICATION_WORK_PROCESSOR_DELAY("replication.work.processor.delay", "0s", PropertyType.TIMEDURATION, "Amount of time to wait before first checking for replication work, not useful outside of tests"), REPLICATION_WORK_PROCESSOR_PERIOD("replication.work.processor.period", "0s", PropertyType.TIMEDURATION, "Amount of time to wait before re-checking for replication work, not useful outside of tests"), http://git-wip-us.apache.org/repos/asf/accumulo/blob/7894feda/docs/src/main/asciidoc/chapters/replication.txt ---------------------------------------------------------------------- diff --git a/docs/src/main/asciidoc/chapters/replication.txt b/docs/src/main/asciidoc/chapters/replication.txt index dc87b62..03bf67b 100644 --- a/docs/src/main/asciidoc/chapters/replication.txt +++ b/docs/src/main/asciidoc/chapters/replication.txt @@ -203,3 +203,112 @@ the implementation of various components operate. |replication.work.assigner | Work Assigner implementation | org.apache.accumulo.master.replication.SequentialWorkAssigner |tserver.replication.batchwriter.replayer.memory| Size of BatchWriter cache to use in applying replication requests | 50M |==== + +=== Example Practical Configuration + +A real-life example is now provided to give concrete application of replication configuration. This +example is a two instance Accumulo system, one primary system and one peer system. They are called +primary and peer, respectively. Each system also have a table of the same name, "my_table". The instance +name for each is also the same (primary and peer), and both have ZooKeeper hosts on a node with a hostname +with that name as well (primary:2181 and peer:2181). + +We want to configure these systems so that "my_table" on "primary" replicates to "my_table" on "peer". + +==== conf/accumulo-site.xml + +We can assign the "unique" name that identifies this Accumulo instance among all others that might participate +in replication together. In this example, we will use the names provided in the description. + +===== Primary + +---- +<property> + <name>replication.name</name> + <value>primary</value> + <description>Defines the unique name</description> +</property> +---- + +===== Peer + +---- +<property> + <name>replication.name</name> + <value>peer</value> +</property> +---- + +==== conf/masters and conf/slaves + +Be *sure* to use non-local IP addresses. Other nodes need to connect to it and using localhost will likely result in +a local node talking to another local node. + +==== Start both instances + +The rest of the configuration is dynamic and is best configured on the fly (in ZooKeeper) than in accumulo-site.xml. + +==== Peer + +The next series of command are to be run on the peer system. Create a user account for the primary instance called +"peer". The password for this account will need to be saved in the configuration on the primary + +---- +root@peer> createtable my_table +root@peer> createuser peer +root@peer> grant -t my_table -u peer Table.WRITE +root@peer> grant -t my_table -u peer Table.READ +root@peer> tables -l +---- + +Remember what the table ID for 'my_table' is. You'll need that to configured the primary instance. + +==== Primary + +Next, configure the primary instance. + +===== Set up the table + +---- +root@primary> createtable my_table +---- + +===== Define the Peer as a replication peer to the Primary + +We're defining the instance with replication.name of 'peer' as a peer. We provide the implementation of ReplicaSystem +that we want to use, and the configuration for the AccumuloReplicaSystem. In this case, the configuration is the Accumulo +Instance name for 'peer' and the ZooKeeper quorum string. The configuration key is of the form +"replication.peer.$peer_name". + +---- +root@primary> config -s replication.peer.peer=org.apache.accumulo.tserver.replication.AccumuloReplicaSystem,peer,$peer_zk_quorum +---- + +===== Set the authentication credentials + +We want to use that special username and password that we created on the peer, so we have a means to write data to +the table that we want to replicate to. The configuration key is of the form "replication.peer.user.$peer_name". + +---- +root@primary> config -s replication.peer.user.peer=peer +root@primary> config -s replication.peer.password.peer=peer +---- + +===== Enable replication on the table + +Now that we have defined the peer on the primary and provided the authentication credentials, we need to configure +our table with the implementation of ReplicaSystem we want to use to replicate to the peer. In this case, our peer +is an Accumulo instance, so we want to use the AccumuloReplicaSystem. + +The configuration for the AccumuloReplicaSystem is the table ID for the table on the peer instance that we +want to replicate into. Be sure to use the correct value for $peer_table_id. The configuration key is of +the form "table.replication.target.$peer_name". + +---- +root@primary> config -t my_table -s table.replication.target.peer=$peer_table_id +---- + +Finally, we can enable replication on this table. + +---- +root@primary> config -t my_table -s table.replication=true +----
