Repository: incubator-griffin Updated Branches: refs/heads/master 0e8852da8 -> 3f86b5a47
Doc/enhancement update some documents Author: younglee1 <[email protected]> Author: jinyli <[email protected]> Closes #263 from younglee1/doc/enhancement. Project: http://git-wip-us.apache.org/repos/asf/incubator-griffin/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-griffin/commit/3f86b5a4 Tree: http://git-wip-us.apache.org/repos/asf/incubator-griffin/tree/3f86b5a4 Diff: http://git-wip-us.apache.org/repos/asf/incubator-griffin/diff/3f86b5a4 Branch: refs/heads/master Commit: 3f86b5a479689887c644f5697a262193a63f5839 Parents: 0e8852d Author: younglee1 <[email protected]> Authored: Wed Apr 25 12:32:46 2018 +0800 Committer: Lionel Liu <[email protected]> Committed: Wed Apr 25 12:32:46 2018 +0800 ---------------------------------------------------------------------- README.md | 10 ++++---- griffin-doc/deploy/deploy-guide.md | 2 +- griffin-doc/dev/dev-env-build.md | 14 ++++------- griffin-doc/docker/griffin-docker-guide.md | 32 +++++++++++++------------ 4 files changed, 27 insertions(+), 31 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-griffin/blob/3f86b5a4/README.md ---------------------------------------------------------------------- diff --git a/README.md b/README.md index de4304c..9220b4f 100644 --- a/README.md +++ b/README.md @@ -27,27 +27,27 @@ Apache Griffin is a model driven data quality solution for modern data systems. ## Getting Started -### First Try of Griffin +### Quick Start You can try Griffin in docker following the [docker guide](griffin-doc/docker/griffin-docker-guide.md). ### Environment for Dev -If you want to develop Griffin, please follow [this document](griffin-doc/dev/dev-env-build.md), to skip complex environment building work. +Follow [Apache Griffin Development Environment Build Guide](griffin-doc/dev/dev-env-build.md) to set up development environment. ### Deployment at Local -If you want to deploy Griffin in your local environment, please follow [this document](griffin-doc/deploy/deploy-guide.md). +If you want to deploy Griffin in your local environment, please follow [Apache Griffin Deployment Guide](griffin-doc/deploy/deploy-guide.md). ## Community -You can access [griffin home page](http://griffin.apache.org). +For more information about Griffin, please visit our website at: [griffin home page](http://griffin.apache.org). You can contact us via email: - dev-list: <a href="mailto:[email protected]";>[email protected]</a> - user-list: <a href="mailto:[email protected]";>[email protected]</a> -You can also subscribe this mail by sending a email to [subscribe dev-list](mailto:[email protected]) and [subscribe user-list](mailto:[email protected]). +You can also subscribe the latest information by sending a email to [subscribe dev-list](mailto:[email protected]) and [subscribe user-list](mailto:[email protected]). You can access our issues on [JIRA page](https://issues.apache.org/jira/browse/GRIFFIN) http://git-wip-us.apache.org/repos/asf/incubator-griffin/blob/3f86b5a4/griffin-doc/deploy/deploy-guide.md ---------------------------------------------------------------------- diff --git a/griffin-doc/deploy/deploy-guide.md b/griffin-doc/deploy/deploy-guide.md index 0693c25..7894e80 100644 --- a/griffin-doc/deploy/deploy-guide.md +++ b/griffin-doc/deploy/deploy-guide.md @@ -18,7 +18,7 @@ under the License. --> # Apache Griffin Deployment Guide -For Griffin users, you can deploy it with some dependencies in your environment, you can follow instructions below. +For Griffin users, please follow the instructions below to deploy Griffin in your environment. Note that there are some dependencies should be installed first. ### Prerequisites You need to install following items http://git-wip-us.apache.org/repos/asf/incubator-griffin/blob/3f86b5a4/griffin-doc/dev/dev-env-build.md ---------------------------------------------------------------------- diff --git a/griffin-doc/dev/dev-env-build.md b/griffin-doc/dev/dev-env-build.md index fc981cb..94e7e8a 100644 --- a/griffin-doc/dev/dev-env-build.md +++ b/griffin-doc/dev/dev-env-build.md @@ -17,17 +17,11 @@ specific language governing permissions and limitations under the License. --> -# Apache Griffin Develop Environment Build Guide -For Griffin developers, you need to build an entire environment for development, it wastes time. -You can directly use the Griffin docker image, to the functional test during your development. -Please follow this [griffin docker guide](../docker/griffin-docker-guide.md). +# Apache Griffin Development Environment Build Guide +We have pre-built Griffin docker images for Griffin developers. You can use the images directly, which should be much faster than building the environment locally. -## Preparation -Follow this [griffin docker guide](../docker/griffin-docker-guide.md#environment-preparation). - -## Start in batch mode or streaming mode -Usually, you can run the docker image in batch mode(always the first choice) or streaming mode(only if you need a streaming environment like kafka data source), choose a mode to start. -For [Batch mode](../docker/griffin-docker-guide.md#how-to-use-griffin-docker-images-in-batch-mode) and [Streaming mode](../docker/griffin-docker-guide.md#how-to-use-griffin-docker-images-in-streaming-mode), follow step 1 and 2, you can start the docker container. +## Set Up with Docker Images +Here are step-by-step instructions on how to [pull Docker images](../docker/griffin-docker-guide.md#environment-preparation) from the repository and run containers using the images. ## Run or Debug at local ### For service module http://git-wip-us.apache.org/repos/asf/incubator-griffin/blob/3f86b5a4/griffin-doc/docker/griffin-docker-guide.md ---------------------------------------------------------------------- diff --git a/griffin-doc/docker/griffin-docker-guide.md b/griffin-doc/docker/griffin-docker-guide.md index bc36759..6038a0a 100644 --- a/griffin-doc/docker/griffin-docker-guide.md +++ b/griffin-doc/docker/griffin-docker-guide.md @@ -20,33 +20,35 @@ under the License. # Apache Griffin Docker Guide Griffin docker images are pre-built on docker hub, users can pull them to try griffin in docker. -## Preparation - -### Environment preparation +## Environment preparation 1. Install [docker](https://docs.docker.com/engine/installation/) and [docker compose](https://docs.docker.com/compose/install/). 2. Increase vm.max_map_count of your local machine, to use elasticsearch. ``` sysctl -w vm.max_map_count=262144 ``` -3. Pull griffin pre-built docker images. - ``` +## Pull the Docker Images + Here lists the griffin and dependent environment docker images: + - `bhlx3lyx7/svc_msr`: This image contains mysql, hadoop, hive, spark, livy, griffin service, griffin measure, and some prepared demo data, it works as a single node spark cluster, providing spark engine and griffin service. + - `bhlx3lyx7/elasticsearch`: This image is based on official elasticsearch, adding some configurations to enable cors requests, to provide elasticsearch service for metrics persist. + - `bhlx3lyx7/kafka`: This image contains kafka 0.8, and some demo streaming data, to provide streaming data source in streaming mode. + - `zookeeper:3.5`: This image is official zookeeper, to provide zookeeper service in streaming mode. + + Pull the images by the following commands. + ``` docker pull bhlx3lyx7/svc_msr:0.2.0 docker pull bhlx3lyx7/elasticsearch docker pull bhlx3lyx7/kafka docker pull zookeeper:3.5 - ``` + ``` Or you can pull the images faster through mirror acceleration if you are in China. - ``` + ``` docker pull registry.docker-cn.com/bhlx3lyx7/svc_msr:0.2.0 docker pull registry.docker-cn.com/bhlx3lyx7/elasticsearch docker pull registry.docker-cn.com/bhlx3lyx7/kafka - docker pull registry.docker-cn.com/zookeeper:3.5 - ``` - The docker images are the griffin environment images. - - `bhlx3lyx7/svc_msr`: This image contains mysql, hadoop, hive, spark, livy, griffin service, griffin measure, and some prepared demo data, it works as a single node spark cluster, providing spark engine and griffin service. - - `bhlx3lyx7/elasticsearch`: This image is based on official elasticsearch, adding some configurations to enable cors requests, to provide elasticsearch service for metrics persist. - - `bhlx3lyx7/kafka`: This image contains kafka 0.8, and some demo streaming data, to provide streaming data source in streaming mode. - - `zookeeper:3.5`: This image is official zookeeper, to provide zookeeper service in streaming mode. + docker pull registry.docker-cn.com/zookeeper:3. + ``` +## Start the Containers +After pulling the images, you can run the docker image in [Batch mode](#how-to-use-griffin-docker-images-in-batch-mode)(always the first choice) or [Streaming mode](#how-to-use-griffin-docker-images-in-streaming-mode)(only if you need a streaming environment like kafka data source). Choose one mode and follow step1&2 to start the docker container. ### How to use griffin docker images in batch mode 1. Copy [docker-compose-batch.yml](https://github.com/apache/incubator-griffin/blob/master/griffin-doc/docker/svc_msr/docker-compose-batch.yml) to your work path. @@ -102,4 +104,4 @@ Griffin docker images are pre-built on docker hub, users can pull them to try gr You can trace the log in streaming-prof.log. ``` tail -f streaming-prof.log - ``` \ No newline at end of file + ```
