This is an automated email from the ASF dual-hosted git repository. szetszwo pushed a commit to branch asf-site-source in repository https://gitbox.apache.org/repos/asf/ratis-hadoop-projects.git
commit a591966b989f3e45d8170c62522ac1e22d48c763 Author: Josh Elser <[email protected]> AuthorDate: Sat Jun 1 21:33:21 2019 -0400 RATIS-581. Add some logservice content to the website Introduces the logservice as a "feature", covers testing. --- content/logservice/_index.md | 13 ++++++++ content/logservice/index.md | 28 ++++++++++++++++ content/logservice/testing/_index.md | 13 ++++++++ content/logservice/testing/docker.md | 63 +++++++++++++++++++++++++++++++++++ content/logservice/testing/index.md | 26 +++++++++++++++ content/logservice/testing/vagrant.md | 24 +++++++++++++ layouts/index.html | 33 ++++++++++-------- 7 files changed, 187 insertions(+), 13 deletions(-) diff --git a/content/logservice/_index.md b/content/logservice/_index.md new file mode 100644 index 0000000..746a26a --- /dev/null +++ b/content/logservice/_index.md @@ -0,0 +1,13 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> diff --git a/content/logservice/index.md b/content/logservice/index.md new file mode 100644 index 0000000..3a000ec --- /dev/null +++ b/content/logservice/index.md @@ -0,0 +1,28 @@ +--- +title: LogService +--- +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +The Ratis LogService is an distributed, log implementation built on top of Apache +Ratis. The LogService is a "recipe" on top of Apache Ratis, providing a higher-level +API as compared to Ratis itself. The LogService provides the ability to create named, +durable, append-only data structures with the ability to perform linear reads. + +Like Ratis, the LogService is designed to be embedded into another application as +a library, as opposed to a standalone daemon. On a confusing note, there are Java +daemons provided for the LogService, but these are solely to be used for testing. + + +* [Testing]({{< ref "testing/index.md" >}}) diff --git a/content/logservice/testing/_index.md b/content/logservice/testing/_index.md new file mode 100644 index 0000000..746a26a --- /dev/null +++ b/content/logservice/testing/_index.md @@ -0,0 +1,13 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> diff --git a/content/logservice/testing/docker.md b/content/logservice/testing/docker.md new file mode 100644 index 0000000..e6fdc13 --- /dev/null +++ b/content/logservice/testing/docker.md @@ -0,0 +1,63 @@ +--- +title: Docker Testing +--- +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +The Docker orchestration is comprised of the following: + +1. A Docker image which has the necessary software to run the LogService +2. A docker-compose.yml file which can launch the necessary containers for a full-fledged LogService "cluster" +3. Scripts to build the Docker image and launch a client environment to interact +with a running cluster via Compose. + +## Building the Docker image + +```bash +$ mvn clean package assembly:single -DskipTests +$ cd ratis-logservice && ./build-docker.sh +``` + +The above will create a Docker image tagged as `ratis-logservice:latest`. + +## Launching a cluster via Compose + +```bash +$ docker-compose up -d +``` + +The Compose orchestration will launch three MetadataServer containers and three +Worker containers, all on the same Docker network. The `-d` option detaches the +container logs from your current shell. + +## Connecting a client + +```bash +$ ./client-env.sh +$ ./bin/shell <...> +$ ./bin/load-test <...> +``` + +The `client-env.sh` script will launch a Docker container which is on the same +network as our cluster running in Compose. + +## Debugging the cluster + +Use `docker logs` to inspect the output from a specific container. You must pass +the name of the container (obtained via `docker-compose ps` or `docker ps`) to +`docker logs`. + +You can also "attach" to a container via `docker exec` to inspect the environment +in which the process is running. Again, using the name of a container obtained as +described above, use `docker exec -it <name> /bin/sh` to attach to the container. diff --git a/content/logservice/testing/index.md b/content/logservice/testing/index.md new file mode 100644 index 0000000..e2ebdbf --- /dev/null +++ b/content/logservice/testing/index.md @@ -0,0 +1,26 @@ +--- +title: LogService Testing +--- +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +The LogService presently has two means for testing itself: Docker with Compose +orchestration and VirtualBox with Vagrant orchestration. + +Docker is suitable for a quick and lightweight orchestration of a full LogService +installation. Vagrant, while heavier-weight that the Docker automation, has the added benefit of being able to leverage [Namazu](http://osrg.github.io/namazu/) for failure +scenarios. Please find more on each using the below references. + +* [Docker]({{< ref "docker.md" >}}) +* [Vagrant]({{< ref "vagrant.md" >}}) diff --git a/content/logservice/testing/vagrant.md b/content/logservice/testing/vagrant.md new file mode 100644 index 0000000..fcac0ed --- /dev/null +++ b/content/logservice/testing/vagrant.md @@ -0,0 +1,24 @@ +--- +title: Vagrant Testing +--- +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +Please refer to the [documentation](https://github.com/apache/incubator-ratis/blob/master/dev-support/vagrant/README.md) for instructions to use the Vagrant automation. + +Starting from the directory `dev-support/vagrant/`: + +* To build all Vagrant boxes, invoke `./run_all_tests.sh build` +* To remove any generated data, invoke `./run_all_tests.sh clean` +* To run the tests, invoke `vagrant resume ratis-servers && vagrant ssh ratis-servers` diff --git a/layouts/index.html b/layouts/index.html index e7d5bcf..da8e288 100644 --- a/layouts/index.html +++ b/layouts/index.html @@ -52,7 +52,7 @@ <div class="container"> <div class="row feature-list"> <h2>Features</h2> - <div class="col-md-4 col-sm-4 feature-item"> + <div class="col-md-6 col-sm-6 feature-item"> <span class="glyphicon glyphicon-transfer" aria-hidden="true"></span> <i class="fa fa-5x fa-fw m-y-3 text-info fa-child"></i> <h4>Pluggable transport</h4> @@ -64,27 +64,34 @@ </p> <p></p> </div> - <div class="col-md-4 col-sm-4 feature-item"> + <div class="col-md-6 col-sm-6 feature-item"> <span class="glyphicon glyphicon-log-in" aria-hidden="true"></span> <h4>Pluggable state machine</h4> <p> Ratis supports a log and state machine. State machine typically contains the data that you want to make highly avialable. Ratis makes it easy to use your own state machine. </p> </div> - <div class="col-md-4 col-sm-4 feature-item"> - <span class="glyphicon glyphicon-tasks" aria-hidden="true"></span> - <h4>Pluggable raft log</h4> - <p>RAFT log is also pluggable, users can provide their own log implementation. The default - implementation - stores log in local files.</p> - <p> - Application can define the polices of how to write data and where it should be written easily. - </p> - </div> </div><!-- /.feature-list --> - + <div class="row feature-list"> + <div class="col-md-6 col-sm-6 feature-item"> + <span class="glyphicon glyphicon-tasks" aria-hidden="true"></span> + <h4>Pluggable raft log</h4> + <p>RAFT log is also pluggable, users can provide their own log implementation. The default + implementation + stores log in local files.</p> + <p> + Application can define the polices of how to write data and where it should be written easily. + </p> + </div> + <div class="col-md-6 col-sm-6 feature-item"> + <span class="glyphicon glyphicon-hdd" aria-hidden="true"></span> + <h4>Log Service</h4> + <p>Ratis provides a log service recipe provides StateMachines to implement a distributed log service with a focused client API. For more information, + please read the <a href="logservice">LogService documentation</a>.</p> + </div> + </div> </div><!-- /.container --> </div><!-- /.feature-list-group -->
