[
https://issues.apache.org/jira/browse/BIGTOP-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158529#comment-14158529
]
Jon Cope edited comment on BIGTOP-1449 at 10/4/14 2:23 PM:
-----------------------------------------------------------
I've validated the smoke-tests.sh with BT-1449 applied. The script just needs
a few small tweaks to smooth it out:
Mapred:
* Err: `java.lang.AssertionError: Can't find hadoop-examples.jar file`
* FIX: smoke-tests.sh: HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce/ from
".../hadoop-mapred"
Hive:
* Success! no changes
Pig:
* Err: undeclared env variable: PIG_HOME
* FIX: add export PIG_HOME=/usr/lib/pig" to smoke-test.sh
Flume:
* add flume to yum install list
* add FLUME_HOME env var
Sqoop:
* same as flume
Other than that, it ran smoothly!
was (Author: jcope):
I've validated the smoke-tests.sh with BT-1449 applied. The script just needs
a few small tweaks to smooth it out:
Mapred:
** Err: `java.lang.AssertionError: Can't find hadoop-examples.jar file`
** FIX: smoke-tests.sh: HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce/ from
".../hadoop-mapred"
Hive:
** Success! no changes
Pig:
** Err: undeclared env variable: PIG_HOME
** FIX: add export PIG_HOME=/usr/lib/pig" to smoke-test.sh
Flume:
** add flume to yum install list
** add FLUME_HOME env var
Sqoop:
** same as flume
Other than that, it ran smoothly!
> Add RC Testing support to vagrant recipe
> ----------------------------------------
>
> Key: BIGTOP-1449
> URL: https://issues.apache.org/jira/browse/BIGTOP-1449
> Project: Bigtop
> Issue Type: Improvement
> Components: vm
> Affects Versions: 0.9.0
> Reporter: jay vyas
> Assignee: jay vyas
> Attachments: BIGTOP-1449.patch
>
>
> We can automatically set up vagrant to install a release candidate on n nodes
> and run smoke tests. Here is how .
> 1) Adding the bigtop folder as a shared folder in Vagrantfile, so that VMs
> have access to bigtop source
> {noformat}
> # the bigtop puppet recipes
> bigtop_puppet = "../../puppet"
> -
> + bigtop_home = "../../../"
> ...
> + bigtop.vm.synced_folder bigtop_home, "/bigtop-home"
> {noformat}
> 2) cd /bigtop-home [ {{sudo puppet apply --modulepath=./ -e "include
> bigtop_toolchain::gradle && alias gradle=/usr/local/gradle/bin/gradle"}}
> 3) Having the bigtop repo in Vagrantfile point at the RC url (modify the line
> in provision.sh
> {{bigtop_yumrepo_uri,http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.8.0/label=centos6/6/artifact/output/}}
> 4) Export HADOOP_CONF_DIR,
> BIGTOP_HOME,HADOOP_MAPRED_HOME,HIVE_HOME,HIVE_CONF_DIR, and so on...
> 5) Ensure that a home dir exists for the user (i.e. as fs superuser (i.e.
> hdfs or root on a posix filesystem) run {{hadoop fs -mkdir /user/vagrant}} )
> 6) Running the tests {{cd bigtop-tests/smoke-tests/ && gradle compileGroovy
> test -Dsmoke-tests=mapreduce --info}}
> Might as well do it so that for next release, we can automate testing of
> bigtop on n-node clusters.
> When doing this task,
> - lets also update {{smoke-tests}} and (1) fix the README (it sais
> smoke-tests, instead of smoke.tests) (2) parameterize ITest . Those are both
> needed for proper automation of RC testing.
> - Also lets bound the amount of hive tests we run. Currently it runs all of
> them. I think we only need it to run "basic" or a few others, and we can
> ensure this by setting system properties like this, so the smoke tests is
> fast and simple:
> {noformat}
> test {
>
> systemProperties["org.apache.bigtop.itest.hivesmoke.TestHiveSmokeBulk.test_include"]="basic"
> }
> {noformat}
> So all in all there are basically two tasks: Add a "test.sh" script to the
> vagrant provisioner which installs toolchain and runs bigtop smoke tests, and
> a few complimentary, minor updates to the smoke-tests .
> *Then, next time an RC comes out, to test, we just update number of nodes and
> run "vagrant up".* :)
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
