Repository: bigtop Updated Branches: refs/heads/master 2a766aa27 -> a991f5a1e
BIGTOP-1365. Updates to Puppet README, and main README.md. Project: http://git-wip-us.apache.org/repos/asf/bigtop/repo Commit: http://git-wip-us.apache.org/repos/asf/bigtop/commit/a991f5a1 Tree: http://git-wip-us.apache.org/repos/asf/bigtop/tree/a991f5a1 Diff: http://git-wip-us.apache.org/repos/asf/bigtop/diff/a991f5a1 Branch: refs/heads/master Commit: a991f5a1ecebd0d6e2602dccae5df459a6f0240d Parents: 2a766aa Author: [email protected] <jayunit100> Authored: Tue Oct 14 10:45:47 2014 -0400 Committer: [email protected] <jayunit100> Committed: Tue Oct 14 10:45:47 2014 -0400 ---------------------------------------------------------------------- README.md | 17 ++++++++++-- bigtop-deploy/puppet/README.md | 55 +++++++++++++++++++++++++++++++++++-- 2 files changed, 67 insertions(+), 5 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/bigtop/blob/a991f5a1/README.md ---------------------------------------------------------------------- diff --git a/README.md b/README.md index c1e24d4..dce5a9b 100644 --- a/README.md +++ b/README.md @@ -65,8 +65,19 @@ Getting Started Below are some recipes for getting started with using Apache Bigtop. As Apache Bigtop has different subprojects, these recipes will continue to evolve. For specific questions it's always a good idea to ping the mailing list at [email protected] to get some immediate feedback, or [open a JIRA](https://issues.apache.org/jira/browse/BIGTOP). -For Users: Running the tests ----------------------------- +For Users: Running the smoke tests. +----------------------------------- + +The simplest way to test bigtop is to: + +* Step 1: Install gradle. +* Step 2: cd bigtop-tests/smoke-tests/ +* Step 3: Follow the instructions in the smoke-tests/README file. + +For integration (API level) testing with maven, read on. + +For Users: Running the integration tests. +----------------------------------------- WARNING: since testing packages requires installing them on a live system it is highly recommended to use VMs for that. Testing Apache Bigtop is done using iTest framework. The tests are organized in maven submodules, with one submodule per Apache Bigtop component. The bigtop-tests/test-execution/smokes/pom.xml defines all submodules to be tested, and each submodule is in its own directory under smokes/, for example: @@ -91,7 +102,7 @@ WARNING: since testing packages requires installing them on a live system it is * Just running hadoop examples, nothing else. - mvn clean verify -D'org.apache.maven-failsafe-plugin.testInclude=**/*TestHadoopExamples*' -f bigtop-tests/test-execution/smokes/package/pom.xml + mvn clean verify -D'org.apache.maven-failsafe-plugin.testInclude=**/*TestHadoopExamples*' -f bigtop-tests/test-execution/smokes/hadoop/pom.xml Note: A minor bug/issue: you need the "testInclude" regular expression above, even if you don't want to customize the tests, since existing test names don't follow the maven integration test naming convention of IT*, but instead, follow the surefire (unit test) convention of Test*. http://git-wip-us.apache.org/repos/asf/bigtop/blob/a991f5a1/bigtop-deploy/puppet/README.md ---------------------------------------------------------------------- diff --git a/bigtop-deploy/puppet/README.md b/bigtop-deploy/puppet/README.md index 0fd56dd..32ac398 100644 --- a/bigtop-deploy/puppet/README.md +++ b/bigtop-deploy/puppet/README.md @@ -1,9 +1,60 @@ # Puppet classes for deploying Hadoop +## Intro + +BigTop currently uses puppet 2.7 (not 3.0). For updates on 3.0 progress, track the +BIGTOP-1365 JIRA. + +Installing and starting hadoop services is non-trivial, and for this reason bigtop + +packages puppet instrumentation for the various ecosystem components, which works + +synergistically with bigtop produced software packages. + +The puppet classes for bigtop deployment setup and deploy hadoop services. + +This includes tasks such as: + +- service installation +- pointing slaves to masters (i.e. regionservers, nodemanagers to their respective master) +- starting the services + +The mode of puppet is masterless : there is no fancy coordination happening behind the scenes. + +Puppet has a notion of a configuration directory, called config. + +When running puppet apply, note that puppet's confdir is *underneath* the --confdir value. + +For example: + +If you have site.csv in /etc/puppet/config, + +Then you should use --confdir=/etc/puppet , and puppet finds the config dir underneath. + +As an end to end example, you can follow the vagrant-puppet recipes to see how to set up + +a puppet managed bigtop hadoop installation. Those examples are gauranteed to work and + +serve as a pedagogical round trip to the way bigtop integrates packaging, deployment, and + +testing all into one package. + +## Debugging + +If in any case, you need to debug these recipes, you can add notify("...") statements into +the puppet scripts. + +In time, we will add more logging and debugging to these recipes. Feel free to submit + +a patch for this ! + ## Configuration -manifests/init.pp expects configuration to live in CSV at $confdir/config/site.csv, -which takes the form +As above, we defined a confdir (i.e. /etc/puppet/) which has a config/ directory in it. + +The heart of puppet is the manifests file. This file ( manifests/init.pp ) + +expects configuration to live in CSV at $confdir/config/site.csv, which takes the form <pre> key,value[,value2,value3]
