Kengo Seki created BIGTOP-3814:
----------------------------------
Summary: Deploying Spark ThriftServer fails because it's launched
too early
Key: BIGTOP-3814
URL: https://issues.apache.org/jira/browse/BIGTOP-3814
Project: Bigtop
Issue Type: Bug
Components: deployment, spark
Reporter: Kengo Seki
I tried to deploy Spark on Rocky 8 using Docker provisioner and came across the
following failure.
{code}
[vagrant@rocky8 docker]$ ./gradlew allclean spark-pkg repo -Dbuildwithdeps=true
[vagrant@rocky8 docker]$ cd provisioner/docker
[vagrant@rocky8 docker]$ ./docker-hadoop.sh -d -C config_rockylinux-8.yaml -r
file:///bigtop-home/output -G -k hdfs,yarn,spark -c 3
...
Notice:
/Stage[main]/Spark::Spark_thriftserver/Package[spark-thriftserver]/ensure:
created
Error: Systemd start for spark-thriftserver failed!
journalctl log for spark-thriftserver:
-- Logs begin at Tue 2022-09-13 10:48:23 UTC, end at Tue 2022-09-13 10:55:42
UTC. --
Sep 13 10:55:35 c93a93f857f8 systemd[1]: Starting LSB: Spark thriftserver...
Sep 13 10:55:35 c93a93f857f8 spark-thriftserver[36957]: Starting Spark
thriftserver (spark-thriftserver):[ OK ]
Sep 13 10:55:35 c93a93f857f8 runuser[36986]: pam_unix(runuser:session): session
opened for user spark by (uid=0)
Sep 13 10:55:35 c93a93f857f8 runuser[36986]: pam_unix(runuser:session): session
closed for user spark
Sep 13 10:55:42 c93a93f857f8 systemd[1]: spark-thriftserver.service: Control
process exited, code=exited status=1
Sep 13 10:55:42 c93a93f857f8 systemd[1]: spark-thriftserver.service: Failed
with result 'exit-code'.
Sep 13 10:55:42 c93a93f857f8 systemd[1]: Failed to start LSB: Spark
thriftserver.
Error:
/Stage[main]/Spark::Spark_thriftserver/Service[spark-thriftserver]/ensure:
change from 'stopped' to 'running' failed: Systemd start for spark-thriftserver
failed!
journalctl log for spark-thriftserver:
-- Logs begin at Tue 2022-09-13 10:48:23 UTC, end at Tue 2022-09-13 10:55:42
UTC. --
Sep 13 10:55:35 c93a93f857f8 systemd[1]: Starting LSB: Spark thriftserver...
Sep 13 10:55:35 c93a93f857f8 spark-thriftserver[36957]: Starting Spark
thriftserver (spark-thriftserver):[ OK ]
Sep 13 10:55:35 c93a93f857f8 runuser[36986]: pam_unix(runuser:session): session
opened for user spark by (uid=0)
Sep 13 10:55:35 c93a93f857f8 runuser[36986]: pam_unix(runuser:session): session
closed for user spark
Sep 13 10:55:42 c93a93f857f8 systemd[1]: spark-thriftserver.service: Control
process exited, code=exited status=1
Sep 13 10:55:42 c93a93f857f8 systemd[1]: spark-thriftserver.service: Failed
with result 'exit-code'.
Sep 13 10:55:42 c93a93f857f8 systemd[1]: Failed to start LSB: Spark
thriftserver.
Notice: /Stage[main]/Spark::Spark_thriftserver/Service[spark-thriftserver]:
Triggered 'refresh' from 3 events
{code}
STS uses HDFS so it should be launched after initializing HDFS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
