[
https://issues.apache.org/jira/browse/BIGTOP-1449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jay vyas updated BIGTOP-1449:
-----------------------------
Description:
We can automatically set up vagrant to install a release candidate on n nodes
and run smoke tests. Here is how .
1) Adding the bigtop folder as a shared folder in Vagrantfile, so that VMs have
access to bigtop source
{noformat}
# the bigtop puppet recipes
bigtop_puppet = "../../puppet"
-
+ bigtop_home = "../../../"
...
+ bigtop.vm.synced_folder bigtop_home, "/bigtop-home"
{noformat}
2) cd /bigtop-home [ {{sudo puppet apply --modulepath=./ -e "include
bigtop_toolchain::gradle && alias gradle=/usr/local/gradle/bin/gradle"}}
3) Having the bigtop repo in Vagrantfile point at the RC url (modify the line
in provision.sh
{{bigtop_yumrepo_uri,http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.8.0/label=centos6/6/artifact/output/}}
4) Export HADOOP_CONF_DIR,
BIGTOP_HOME,HADOOP_MAPRED_HOME,HIVE_HOME,HIVE_CONF_DIR, and so on...
5) Ensure that a home dir exists for the user (i.e. as fs superuser (i.e. hdfs
or root on a posix filesystem) run {{hadoop fs -mkdir /user/vagrant}} )
6) Running the tests {{cd bigtop-tests/smoke-tests/ && gradle compileGroovy
test -Dsmoke-tests=mapreduce --info}}
Might as well do it so that for next release, we can automate testing of
bigtop on n-node clusters.
When doing this task,
- lets also update {{smoke-tests}} and (1) fix the README (it sais smoke-tests,
instead of smoke.tests) (2) parameterize ITest . Those are both needed for
proper automation of RC testing.
- Also lets bound the amount of hive tests we run. Currently it runs all of
them. I think we only need it to run "basic" or a few others, and we can
ensure this by setting system properties like this, so the smoke tests is fast
and simple:
{noformat}
test {
systemProperties["org.apache.bigtop.itest.hivesmoke.TestHiveSmokeBulk.test_include"]="basic"
}
{noformat}
- *Then, next time an RC comes out, to test, we just update number of nodes and
run "vagrant up".* :)
was:
We can automatically set up vagrant to install a release candidate on n nodes
and run smoke tests. Here is how .
1) Adding the bigtop folder as a shared folder in Vagrantfile, so that VMs have
access to bigtop source
{noformat}
# the bigtop puppet recipes
bigtop_puppet = "../../puppet"
-
+ bigtop_home = "../../../"
...
+ bigtop.vm.synced_folder bigtop_home, "/bigtop-home"
{noformat}
2) cd /bigtop-home [ {{sudo puppet apply --modulepath=./ -e "include
bigtop_toolchain::gradle && alias gradle=/usr/local/gradle/bin/gradle"}}
3) Having the bigtop repo in Vagrantfile point at the RC url (modify the line
in provision.sh
{{bigtop_yumrepo_uri,http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.8.0/label=centos6/6/artifact/output/}}
4) Export HADOOP_CONF_DIR,
BIGTOP_HOME,HADOOP_MAPRED_HOME,HIVE_HOME,HIVE_CONF_DIR, and so on...
5) Ensure that a home dir exists for the user (i.e. as fs superuser (i.e. hdfs
or root on a posix filesystem) run {{hadoop fs -mkdir /user/vagrant}} )
6) Running the tests {{cd bigtop-tests/smoke-tests/ && gradle compileGroovy
test -Dsmoke-tests=mapreduce --info}}
Might as well do it so that for next release, we can automate testing of
bigtop on n-node clusters.
When doing this task,
- lets also update {{smoke-tests}} and (1) fix the README (it sais smoke-tests,
instead of smoke.tests) (2) parameterize ITest . Those are both needed for
proper automation of RC testing.
- Also lets bound the amount of hive tests we run. Currently it runs all of
them. I think we only need it to run "basic" or a few others, and we can
ensure this by setting system properties.
*Then, next time an RC comes out, to test, we just update number of nodes and
run "vagrant up".* :)
> Add RC Testing support to vagrant recipe
> ----------------------------------------
>
> Key: BIGTOP-1449
> URL: https://issues.apache.org/jira/browse/BIGTOP-1449
> Project: Bigtop
> Issue Type: Improvement
> Components: vm
> Affects Versions: 0.9.0
> Reporter: jay vyas
>
> We can automatically set up vagrant to install a release candidate on n nodes
> and run smoke tests. Here is how .
> 1) Adding the bigtop folder as a shared folder in Vagrantfile, so that VMs
> have access to bigtop source
> {noformat}
> # the bigtop puppet recipes
> bigtop_puppet = "../../puppet"
> -
> + bigtop_home = "../../../"
> ...
> + bigtop.vm.synced_folder bigtop_home, "/bigtop-home"
> {noformat}
> 2) cd /bigtop-home [ {{sudo puppet apply --modulepath=./ -e "include
> bigtop_toolchain::gradle && alias gradle=/usr/local/gradle/bin/gradle"}}
> 3) Having the bigtop repo in Vagrantfile point at the RC url (modify the line
> in provision.sh
> {{bigtop_yumrepo_uri,http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.8.0/label=centos6/6/artifact/output/}}
> 4) Export HADOOP_CONF_DIR,
> BIGTOP_HOME,HADOOP_MAPRED_HOME,HIVE_HOME,HIVE_CONF_DIR, and so on...
> 5) Ensure that a home dir exists for the user (i.e. as fs superuser (i.e.
> hdfs or root on a posix filesystem) run {{hadoop fs -mkdir /user/vagrant}} )
> 6) Running the tests {{cd bigtop-tests/smoke-tests/ && gradle compileGroovy
> test -Dsmoke-tests=mapreduce --info}}
> Might as well do it so that for next release, we can automate testing of
> bigtop on n-node clusters.
> When doing this task,
> - lets also update {{smoke-tests}} and (1) fix the README (it sais
> smoke-tests, instead of smoke.tests) (2) parameterize ITest . Those are both
> needed for proper automation of RC testing.
> - Also lets bound the amount of hive tests we run. Currently it runs all of
> them. I think we only need it to run "basic" or a few others, and we can
> ensure this by setting system properties like this, so the smoke tests is
> fast and simple:
> {noformat}
> test {
>
> systemProperties["org.apache.bigtop.itest.hivesmoke.TestHiveSmokeBulk.test_include"]="basic"
> }
> {noformat}
> - *Then, next time an RC comes out, to test, we just update number of nodes
> and run "vagrant up".* :)
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
