Hi Jay,
I looked at the README in master, and it has since been updated. It was the
README in the 0.7.0 distro that was the issue.
I tried running the gradle smokes in master and I am getting this:
:pig:test
ENV VARIABLE: PIG_HOME = /usr/lib/pig
org.apache.bigtop.itest.hadoop.mapreduce.TestPigSmoke > test FAILED
java.lang.AssertionError at TestPigSmoke.groovy:56
1 test completed, 1 failed
:pig:test FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':pig:test'.
> There were failing tests. See the report at:
> file:///root/git/bigtop/bigtop-tests/smoke-tests/pig/build/reports/tests/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output.
BUILD FAILED
Total time: 32.62 secs
From: jay vyas <[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Tuesday, September 23, 2014 1:43 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: smoke tests in 0.7.0
Hi tim ,
1) thanks for noting. looks like time for a README update. the "package" part
needs to be modified . could you create a JIRA for that?
2) Also, iirc , is there a /user/root/ directory in hdfs? There needs to be
one so that it can write the local files.
weve also completed a simpler way of running the tests recently in the
"smoke-tests" package which you can try (see the README).
On Tue, Sep 23, 2014 at 4:31 PM, Tim Harsch
<[email protected]<mailto:[email protected]>> wrote:
Hi all,
I am having trouble getting the smoke tests to run in 0.7.0. I guess my first
question is: which systems are these smokes being validated against at release
time? I'm wondering if it should work for CentOS6
For installing bigtop, I followed instructions in the Hadoop for Dummies book.
I install a CentOS6 vmWare player. And installed bigtop via the following:
"yum install hadoop\* mahout\* oozie\* hbase\* hive\* hue\* pig\* zookeeper\*
giraph\*"
I believe I now have a working hadoop system, "jps" confirms running services.
But, almost none of the tests run:
I've read the top level README, and followed the wiki
https://cwiki.apache.org/confluence/display/BIGTOP/Running+integration+and+system+tests.
README:
Following the first step for testing yields:
[root@localhost bigtop-0.7.0]# mvn install -DskipTests -DskipITs
-DperformRelease -f bigtop-tests/test-execution/smokes/package/pom.xml
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project
(/root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/package/pom.xml)
has 1 error
[ERROR] Non-readable POM
/root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/package/pom.xml:
/root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/package/pom.xml
(No such file or directory)
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
>From the error we see there is no package module.
WIKI:
Basically advises to run a submodule… hadoop seems like a good one. If I run
it though I get 8 of 8 tests failing. Here is the first sign of trouble:
Running org.apache.bigtop.itest.hadoop.mapreduce.TestHadoopSmoke
Failed command: hadoop fs -rmr test.hadoopsmoke.1411495413183/cachefile/out
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-streaming.jar -D mapred.map.tasks=1
-D mapred.reduce.tasks=1 -D mapred.job.name<http://mapred.job.name>=Experiment
-cacheArchive
file:////user/root/test.hadoopsmoke.1411495413183/cachefile/cachedir.jar#testlink
-input test.hadoopsmoke.1411495413183/cachefile/input.txt -mapper map.sh -file
map.sh -reducer cat -output test.hadoopsmoke.1411495413183/cachefile/out
-verbose
error code: 2
stdout: [2014-09-23 20:03:38,522 WARN [main] streaming.StreamJob
(StreamJob.java:parseArgv(290)) - -file option is deprecated, please use
generic option -files instead., 2014-09-23 20:03:38,913 WARN [main]
streaming.StreamJob (StreamJob.java:parseArgv(337)) - -cacheArchive option is
deprecated, please use -archives instead., STREAM:
addTaskEnvironment=HADOOP_ROOT_LOGGER=, STREAM:
shippedCanonFiles_=[/root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/hadoop/target/map.sh],
STREAM: shipped: true
/root/bigtop/bigtop-0.7.0/bigtop-tests/test-execution/smokes/hadoop/target/map.sh,
STREAM: cmd=map.sh, STREAM: cmd=null, STREAM: shipped:
Seems like I must have something wrong.
Any help appreciated.
Thanks,
Tim
--
jay vyas
