[
https://issues.apache.org/jira/browse/BIGTOP-1222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072234#comment-14072234
]
Dawson Choong edited comment on BIGTOP-1222 at 7/24/14 6:05 PM:
----------------------------------------------------------------
Yes, I've been working on a parametrization patch. Basically what I did was
write a function testSelector() that grabs the available directories to be
tested, such as the ones listed in bigtop-smoke-tests. testSelector will then
generate tasks called "test-artifact" based on the user's command line input.
You can run individual tests such as: gradle clean compileGroovy test-pig
test-pig test-sqoop. Here's a snippet of what I got so far:
{code}
def testSelector = {
File srcDir
srcDir = file("${BASE_DIR}")
def artifactFiles = files {srcDir.listFiles()}
def artifactList = []
artifactFiles.each { File file ->
artifactList.add(file.name)
}
artifactList.each { artifact ->
task "test-${artifact}" << {
artifactsToTest.add("${artifact}")
}
}
printTestEnv()
}
{code}
The functions then adds all the artifacts specified on command line to an array
artifactsToTest. This array can be used for anything. Right now I'm using it
for printTestEnv(), which is a function that does the same thing as
[~jayunit100]'s test(), except it prints the environments of each object in
artifactsToTest. This way you won't have to comment/uncomment the objects you
want/don't want each time you test. The only issue with my implementation is
that the only things you can test are the directories that are available in
bigtop-smoke-tests (ie conf, flume, pig, sqoop). Therefore, as of now, objects
like "HADOOP_MAPRED_HOME" can't be used.
Let me know your thoughts.
was (Author: dawson.choong):
Yes, I've been working on a parametrization patch. Basically what I did was
write a function testSelector() that grabs the available directories to be
tested, such as the ones listed in bigtop-smoke-tests. testSelector will then
generate tasks called "test-artifact" based on the user's command line input.
Here's a snippet of what I got so far:
{code}
def testSelector = {
File srcDir
srcDir = file("${BASE_DIR}")
def artifactFiles = files {srcDir.listFiles()}
def artifactList = []
artifactFiles.each { File file ->
artifactList.add(file.name)
}
artifactList.each { artifact ->
task "test-${artifact}" << {
artifactsToTest.add("${artifact}")
}
}
printTestEnv()
}
{code}
The functions then adds all the artifacts specified on command line to an array
artifactsToTest. This array can be used for anything. Right now I'm using it
for printTestEnv(), which is a function that does the same thing as
[~jayunit100]'s test(), except it prints the environments of each object in
artifactsToTest. This way you won't have to comment/uncomment the objects you
want/don't want each time you test. The only issue with my implementation is
that the only things you can test are the directories that are available in
bigtop-smoke-tests (ie conf, flume, pig, sqoop). Therefore, as of now, objects
like "HADOOP_MAPRED_HOME" can't be used.
Let me know your thoughts.
> Simplify and gradleize a subset of the bigtop smokes
> ----------------------------------------------------
>
> Key: BIGTOP-1222
> URL: https://issues.apache.org/jira/browse/BIGTOP-1222
> Project: Bigtop
> Issue Type: Improvement
> Components: Build, Tests
> Affects Versions: 0.7.0
> Reporter: jay vyas
> Assignee: jay vyas
> Fix For: backlog
>
> Attachments: BIGTOP-1222-2.patch, BIGTOP-1222.patch,
> BIGTOP-1222.patch, BIGTOP-1222.patch, BIGTOP-1222.patch, BIGTOP-1222.patch,
> BIGTOP-1222.patch
>
>
> (Rewritten the description for clarity)
> We need an easier way to run bigtop smoke tests, and gradle provides this:
> 1) Easy to script/modify
> 2) Human readable
> 3) equally oriented towards both groovy and plain old java
> The advantage of this method to running smokes :
> 1) No need to compile a jar : this is a costly step and not much value added,
> also creates indirection which can make debugging a broken test very hard.
> 2) Simple: A smoke test doesnt need to make low level API calls or be
> compiled against the right APIs - rather, it should test the end user
> interface ("hive -q ....", "pig -x ....", "hadoop jar ....", and so on).
> 3) Customizable: The smoke tests shouldnt require users to have to write XML
> and debug environmental variables / grep around for System properties etc.
> Rather, a high level controller should do all that checking for you.
> The initial idea was to write a python/bash implementation wrapper of
> scripts, but that was replaced by the idea of using gradle. The advantage of
> gradle is that we don't need to manually set the classpath and run groovy
> commands: Gradle wraps groovy scripts in their native java context quite
> nicely - but it doesnt add any other unnecessary overhead (xml, jar files, no
> need for complex xml tag wrappers for simple tasks - just plain groovy code).
> So, here the goal is just to create a nice, clean, extensible non-jar,
> non-API dependent gradle runner for the smoke tests which exersizes the
> hadoop cluster the same way a typical end-user would.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
