Kengo Seki created BIGTOP-3415:
----------------------------------
Summary: Extend timeout for Spark smoke test
Key: BIGTOP-3415
URL: https://issues.apache.org/jira/browse/BIGTOP-3415
Project: Bigtop
Issue Type: Improvement
Components: spark, tests
Reporter: Kengo Seki
Assignee: Kengo Seki
Regarding Spark smoke test, testSparkSQL succeeded but testSparkR was timed out
in some cases.
https://ci.bigtop.apache.org/job/Bigtop-trunk-smoke-tests/166/COMPONENTS=hdfs.yarn.spark@spark,OS=centos-8-x86_64-deploy/console
https://ci.bigtop.apache.org/job/Bigtop-trunk-smoke-tests/166/COMPONENTS=hdfs.yarn.spark@spark,OS=debian-9-amd64-deploy/console
https://ci.bigtop.apache.org/job/Bigtop-trunk-smoke-tests/166/COMPONENTS=hdfs.yarn.spark@spark,OS=debian-9-arm64-deploy/console
https://ci.bigtop.apache.org/job/Bigtop-trunk-smoke-tests/166/COMPONENTS=hdfs.yarn.spark@spark,OS=debian-10-amd64-deploy/console
https://ci.bigtop.apache.org/job/Bigtop-trunk-smoke-tests/166/COMPONENTS=hdfs.yarn.spark@spark,OS=fedora-31-aarch64-deploy/console
https://ci.bigtop.apache.org/job/Bigtop-trunk-smoke-tests/166/COMPONENTS=hdfs.yarn.spark@spark,OS=fedora-31-x86_64-deploy/console
https://ci.bigtop.apache.org/job/Bigtop-trunk-smoke-tests/166/COMPONENTS=hdfs.yarn.spark@spark,OS=ubuntu-18.04-amd64-deploy/console
I tried to run the same tests on a 4C8T/32GiB x86_64 machine locally. Both
tests succeeded before timeout and testSparkR took 1 minute and 30 seconds.
So current timeout setting (120 seconds) may be a bit severe for the smoke test
environment (m3.xlarge EC2 instance, 4vCPU/15GiB memory).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
