guyuqi commented on PR #1060:
URL: https://github.com/apache/bigtop/pull/1060#issuecomment-1331922520
@kevinw66
Sucessfully built Spark on `Rockylinux8` and `Debian11-arm64`.
The smoke tests are also passed on these Distros respectively.
**But** It **failed** to run Spark smoke tests on Fedora-36:
```
org.apache.bigtop.itest.spark.TestSpark > testSparkSQL STANDARD_OUT
SPARK_MASTER isn't set. yarn will be used as the master URL. Refer to
smoke-tests/README If this isn't what you you expect.
Failed command: timeout 300 /usr/lib/spark/bin/spark-shell --master yarn
--class org.apache.spark.examples.sql.SparkSQLExample --jars
/usr/lib/spark/examples/jars/spark-examples*.jar >
/tmp/TestSpark_testSparkSQL.log 2>&1
error code: 124
stdout: []
stderr: []
org.apache.bigtop.itest.spark.TestSpark > testSparkSQL FAILED
java.lang.AssertionError: Failed ...
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert$assertTrue$0.callStatic(Unknown Source)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:196)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:216)
at
org.apache.bigtop.itest.spark.TestSpark.testSparkSQL(TestSpark.groovy:91)
org.apache.bigtop.itest.spark.TestSpark > testSparkR STANDARD_OUT
SPARK_MASTER isn't set. yarn will be used as the master URL. Refer to
smoke-tests/README If this isn't what you you expect.
Failed command: timeout 300 /usr/lib/spark/bin/spark-submit --master
yarn /tmp/dataframe.R > /tmp/TestSpark_testSparkR.log 2>&1
error code: 124
stdout: []
stderr: []
org.apache.bigtop.itest.spark.TestSpark > testSparkR FAILED
java.lang.AssertionError: Failed to execute SparkR script
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert$assertTrue$0.callStatic(Unknown Source)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:196)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:216)
at
org.apache.bigtop.itest.spark.TestSpark.testSparkR(TestSpark.groovy:113)
Gradle Test Executor 2 finished executing tests.
> Task :bigtop-tests:smoke-tests:spark:test FAILED
```
**Rocky8:**
```
$ ./docker-hadoop.sh --create 3 --image bigtop/puppet:trunk-rockylinux-8 \
--memory 16g --repo file:///bigtop-home/output --disable-gpg-check \
--enable-local-repo \
--stack hdfs,yarn,mapreduce,spark \
--smoke-tests spark
```
```
> Task :bigtop-tests:smoke-tests:spark:test
Finished generating test XML results (0.011 secs) into:
/bigtop-home/bigtop-tests/smoke-tests/spark/build/test-results/test
Generating HTML test report...
Finished generating test html results (0.022 secs) into:
/bigtop-home/bigtop-tests/smoke-tests/spark/build/reports/tests/test
Now testing...
:bigtop-tests:smoke-tests:spark:test (Thread[Execution worker for ':' Thread
15,5,main]) completed. Took 1 mins 35.486 secs.
BUILD SUCCESSFUL in 2m 39s
```
**Debian-11**
`config_debian-11.yaml`
```
components: [hdfs, yarn, mapreduce, spark]
enable_local_repo: true
smoke_test_components: [spark]
```
```
$ uname -a
5.13.0-40-generic #45~20.04.1-Ubuntu SMP Mon Apr 4 09:40:54 UTC 2022 aarch64
aarch64 aarch64 GNU/Linux
$ ./docker-hadoop.sh -C config_debian-11.yaml -c 3 -s -d
.....
...
..
Gradle Test Executor 2 finished executing tests.
> Task :bigtop-tests:smoke-tests:spark:test
Finished generating test XML results (0.027 secs) into:
/bigtop-home/bigtop-tests/smoke-tests/spark/build/test-results/test
Generating HTML test report...
Finished generating test html results (0.043 secs) into:
/bigtop-home/bigtop-tests/smoke-tests/spark/build/reports/tests/test
Now testing...
:bigtop-tests:smoke-tests:spark:test (Thread[Execution worker for ':' Thread
8,5,main]) completed. Took 2 mins 21.73 secs.
BUILD SUCCESSFUL in 3m 32s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
