This is an automated email from the ASF dual-hosted git repository.
ywkim pushed a commit to branch cnb
in repository https://gitbox.apache.org/repos/asf/bigtop.git
The following commit(s) were added to refs/heads/cnb by this push:
new e1cd59d BIGTOP-3241: Smoke test for Zookeeper on Kubernetes
e1cd59d is described below
commit e1cd59de8619dc225d3054c26d7da40e23ceb0de
Author: Youngwoo Kim <[email protected]>
AuthorDate: Tue Oct 15 13:08:34 2019 +0900
BIGTOP-3241: Smoke test for Zookeeper on Kubernetes
---
README.md | 13 +++++++++
bigtop-tests/smoke-tests/README | 31 +++++-----------------
.../smoke-tests/zookeeper/TestZookeeper.groovy | 9 ++++---
3 files changed, 26 insertions(+), 27 deletions(-)
diff --git a/README.md b/README.md
index 77b24c6..53f07d8 100755
--- a/README.md
+++ b/README.md
@@ -129,6 +129,19 @@ $ kubectl exec -n $NS zookeeper-0 -- bin/zkServer.sh status
```
Refer to https://github.com/helm/charts/tree/master/incubator/zookeeper for
more configurations.
+Getting Started
+===============
+
+Below are some recipes for getting started with using Apache Bigtop. As Apache
Bigtop has different subprojects, these recipes will continue to evolve.
+For specific questions it's always a good idea to ping the mailing list at
[email protected] to get some immediate feedback, or [open a
JIRA](https://issues.apache.org/jira/browse/BIGTOP).
+
+For Users: Running the smoke tests
+-----------------------------------
+
+The simplest way to test bigtop is described in
bigtop-tests/smoke-tests/README file
+
+For integration (API level) testing with maven, read on.
+
# Cloud Native Bigtop
This is the content for the talk given by jay vyas and sid mani @ apachecon
2019 in Las Vegas, you can watch it here
https://www.youtube.com/watch?v=LUCE63q !
diff --git a/bigtop-tests/smoke-tests/README b/bigtop-tests/smoke-tests/README
index 777948f..e4c9ae2 100644
--- a/bigtop-tests/smoke-tests/README
+++ b/bigtop-tests/smoke-tests/README
@@ -1,9 +1,9 @@
-# This is the new smoke testing module for bigtop
+# This is the new smoke testing module for Bigtop
## Goals
This gradle based test framework aims at making it extremely simple to
-customize and test Apache BigTop's hadoop distribution.
+customize and test Apache BigTop's distribution.
- Easily add new tests to bigtop without adding new directories / xml files.
- Debug tests easily and make logging extremely transparent.
@@ -17,27 +17,10 @@ So overall, this framework makes testing with apache bigtop
easy as well as fun!
You need to export several environment variables. For example,
```
-export HADOOP_CONF_DIR=/etc/hadoop/conf/
-export HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce/
-export HIVE_HOME=/usr/lib/hive/
-export PIG_HOME=/usr/lib/pig/
-export FLUME_HOME=/usr/lib/flume/
-export HIVE_CONF_DIR=/etc/hive/conf/
export JAVA_HOME="/usr/lib/jvm/java-openjdk/"
-export MAHOUT_HOME="/usr/lib/mahout"
-export SPARK_HOME="/usr/lib/spark"
-export TAJO_HOME="/usr/lib/tajo"
-export OOZIE_TAR_HOME="/usr/lib/oozie/doc"
-```
-Fallback mode for shark-shell submission is 'yarn-client'. For a good reason,
-we aren't using YARN to provision cluster services. Hence, in order to execute
-the test against a standalone Spark cluster, like we deploy would require two
-more environment variables:
-`
-export SPARK_MASTER_IP=bigtop1.docker
-export SPARK_MASTER_PORT=7077
-`
+```
+If you are running smoke tests on Kubernetes, you need ```kubectl``` in your
PATH variable.
Then, simply invoke the tests you want to run, as described below.
@@ -46,17 +29,17 @@ Invoking test task of flume subproject will execute flume
tests. To test hive as
you would add the invocation of hive:test, and so on. To run the tests from
the top-level
Bigtop folder, follow the example below:
```
- ./gradlew bigtop-tests:smoke-tests:flume:test
bigtop-tests:smoke-tests:hive:test -Psmoke.tests --info
+ ./gradlew bigtop-tests:smoke-tests:zookeeper:test
bigtop-tests:smoke-tests:rook-ceph:test -Psmoke.tests --info
```
You also can run smoke tests from bigtop-tests/smoke-tests directory,
or from any subproject of it. E.g.
```
cd bigtop-tests/smoke-tests
- ../../gradlew hive:test -Psmoke.tests
+ ../../gradlew zookeeper:test -Psmoke.tests
```
or
```
- cd bigtop-tests/smoke-tests/hive
+ cd bigtop-tests/smoke-tests/zookeeper
../../../gradlew test -Psmoke.tests
```
Instead of using top-level Gradle wrapper one can also use system-wide
diff --git a/bigtop-tests/smoke-tests/zookeeper/TestZookeeper.groovy
b/bigtop-tests/smoke-tests/zookeeper/TestZookeeper.groovy
index 28be70a..0c95241 100644
--- a/bigtop-tests/smoke-tests/zookeeper/TestZookeeper.groovy
+++ b/bigtop-tests/smoke-tests/zookeeper/TestZookeeper.groovy
@@ -31,6 +31,8 @@ class TestZookeeper {
static private Log LOG = LogFactory.getLog(Object.class)
static Shell sh = new Shell("/bin/bash -s")
+ static String DEFAULT_NAMESPACE = "bigtop";
+ static String DEFAULT_POD_NAME = "zookeeper-0";
@BeforeClass
static void setUp() {
@@ -46,12 +48,13 @@ class TestZookeeper {
void testZkServerStatus() {
// Basic test to verify that the server is running, and is in a
// state that we expect.
- LOG.info('Running zkServer.sh status');
- sh.exec("/usr/lib/zookeeper/bin/zkServer.sh status");
+ LOG.info('Running zkServer.sh status on pod');
+ sh.exec("kubectl exec -n " + DEFAULT_NAMESPACE + " " + DEFAULT_POD_NAME +
" -- bin/zkServer.sh status");
logError(sh);
assertTrue("Failed ...", sh.getRet() == 0);
- String out = sh.getOut()[0].trim();
+ String out = sh.getOut().toString();
+ LOG.info("Output of zkServer.sh status:\n" + out);
assertTrue(out.contains("Mode"));
// If this is the only Zookeeper node, then we should be in
// "standalone" mode. If not, we should be in "leader" or
