[
https://issues.apache.org/jira/browse/BIGTOP-1222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993364#comment-13993364
]
jay vyas edited comment on BIGTOP-1222 at 5/10/14 3:44 AM:
-----------------------------------------------------------
Made some progress with a raw gradle test runner which runs pure groovy
scripts, no jar required (couple of hacks required to comment out the jar copy
resource directive and replace with a manual copy of the input tests files for
TestHadoopExamples.groovy, but those will be easy to solve with an try/catch
block update to TestHadoopExamples.groovy which i can put in the final patch.
Very raw, just a prototype of how it might work.
This snippet runs the hadoop examples tests, but you have to manually copy over
data from the resources first and "comment out" the jar oriented build
utilities hooks which extract the files from the jar.
{noformat}
apply plugin: 'groovy'
repositories {
mavenCentral()
}
dependencies {
//needed to avoid groovy not on classpath error.
testCompile module('org.codehaus.groovy:groovy:1.8.0')
testCompile group:'org.apache.bigtop.itest', name:'itest-common',
version:'0.7.0',transitive:'true'
testCompile group:'org.apache.hadoop',name:'hadoop-common',
version:'2.0.6-alpha',transitive:'false'
testCompile group:'org.apache.hadoop',name:'hadoop-hdfs',
version:'2.1.0-beta',transitive:'false'
}
sourceSets {
test {
/**
* This will put the input files into the jar.
* TODO: We should refactor tests, over time, not to require running
from jar.
*/
resources {
srcDirs =
['/opt/bigtop/bigtop-tests/test-artifacts/hadoop/src/main/resources/',
'conf/']
}
groovy {
srcDirs = ['/opt/bigtop/bigtop-tests/test-artifacts/hadoop/']
exclude 'src/main/groovy/org/apache/bigtop/itest/hadoop/hdfs/**'
exclude 'src/main/groovy/org/apache/bigtop/itest/hadoop/hdfs/**'
exclude 'src/main/groovy/org/apache/bigtop/itest/hadoop/hcfs/**'
exclude 'src/main/groovy/org/apache/bigtop/itest/hadoop/yarn/**'
}
}
}
test.doFirst {
def vars = ["HADOOP_MAPRED_HOME"]
vars.each() {
def value = System.getenv("${it}")
if(value == null || value == "null")
throw new GradleException("undeclared env variable: ${it}")
else
println("${it} = $value");
}
}
{noformat}
was (Author: jayunit100):
Made some progress with a raw gradle test runner which runs pure groovy
scripts, no jar required (couple of hacks required to comment out the jar copy
resource directive and replace with a manual copy of the input tests files for
TestHadoopExamples.groovy, but those will be easy to solve with an try/catch
block update to TestHadoopExamples.groovy which i can put in the final patch.
Very raw, just a prototype of how it might work.
This snippet runs the hadoop examples tests, but you have to manually copy over
data from the resources first and "comment out" the jar oriented build
utilities hooks which extract the files from the jar.
{noformat}
apply plugin: 'groovy'
repositories {
mavenCentral()
}
dependencies {
//needed to avoid groovy not on classpath error.
testCompile module('org.codehaus.groovy:groovy:1.8.0')
testCompile 'org.apache.bigtop.itest:itest-common:0.7.0'
testCompile 'org.apache.hadoop:hadoop-common:2.0.6-alpha'
testCompile 'org.apache.hadoop:hadoop-hdfs:2.1.0-beta'
}
sourceSets {
test {
/**
* This will put the input files into the jar.
* TODO: We should refactor tests, over time, not to require running
from jar.
*/
resources {
srcDirs =
['/opt/bigtop/bigtop-tests/test-artifacts/hadoop/src/main/resources/']
}
groovy {
srcDirs = ['/opt/bigtop/bigtop-tests/test-artifacts/hadoop/']
exclude 'src/main/groovy/org/apache/bigtop/itest/hadoop/hdfs/**'
exclude 'src/main/groovy/org/apache/bigtop/itest/hadoop/hcfs/**'
exclude 'src/main/groovy/org/apache/bigtop/itest/hadoop/yarn/**'
}
}
}
test.doFirst {
def vars = ["HADOOP_MAPRED_HOME"]
vars.each() {
def value = System.getenv("${it}")
if(value == null || value == "null")
throw new GradleException("undeclared env variable: ${it}")
else
println("${it} = $value");
}
}
{noformat}
> Simplify and gradleize a subset of the bigtop smokes
> ----------------------------------------------------
>
> Key: BIGTOP-1222
> URL: https://issues.apache.org/jira/browse/BIGTOP-1222
> Project: Bigtop
> Issue Type: Improvement
> Components: Build
> Affects Versions: 0.7.0
> Reporter: jay vyas
> Fix For: backlog
>
>
> Currently, there is a JIRA underway to make running the maven based smoke
> tests easier: BIGTOP-1195.
> Eventually, however, maybe we could run these smokes from gradle. I think
> that will obviate BIGTOP-1195 (Although i still assert a bash driver is a big
> win/gain for bigtop's goals : which are to unify the hadoop packaging and
> deployment paradigm).
> - run the smokes using a simple gradle goal
> - smokes should be easily runnable as scripts, with no need for jar file
> intermediates.
> - The bash driver for BIGTOP-1195 (if accepted, still under debate) should be
> upgraded to use the new gradle smokes
> - Delete old maven smokes.
> This might be a little ambitious, if so others chime in. I'm not a
> gradle/groovy expert but getting more well versed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
