YalikWang commented on PR #7791: URL: https://github.com/apache/seatunnel/pull/7791#issuecomment-2408477189
@Hisoka-X E2E test failed with Spark Engine because of time-out. After submitting to Spark Engine, it seems that the job did not run. Could you please help me to analysis the reason of it ? Or should I disable spark engine container? e2e test console log : 63392 [docker-java-stream-1626631782] INFO 🐳 [tyrantlucifer/spark:3.3.0] - STDOUT: 24/10/12 09:25:23 INFO Utils: Successfully started service 'MasterUI' on port 8080. 63454 [main] INFO org.apache.seatunnel.e2e.common.container.AbstractTestContainer - Execute command in container[tyrantlucifer/spark:3.3.0] ==================== Shell Command start ==================== /tmp/seatunnel/bin/start-seatunnel-spark-3-connector-v2.sh --config /tmp/rocketmq-sink_fake_to_rocketmq_multi_tables.conf --name rocketmq-sink_fake_to_rocketmq_multi_tables.conf --master local --deploy-mode client ==================== Shell Command end ==================== 63485 [docker-java-stream-1626631782] INFO 🐳 [tyrantlucifer/spark:3.3.0] - STDOUT: 24/10/12 09:25:23 INFO MasterWebUI: Bound MasterWebUI to 0.0.0.0, and started at http://39bd998c4ce0:8080 63793 [docker-java-stream-1626631782] INFO 🐳 [tyrantlucifer/spark:3.3.0] - STDOUT: 24/10/12 09:25:24 INFO Master: I have been elected leader! New state: ALIVE spark docker logs: 09:25:19.92 09:25:19.93 Welcome to the Bitnami spark container 09:25:19.93 Subscribe to project updates by watching https://github.com/bitnami/containers 09:25:19.93 Submit issues and feature requests at https://github.com/bitnami/containers/issues 09:25:19.93 09:25:19.94 INFO ==> ** Starting Spark setup ** realpath: /bitnami/spark/conf: No such file or directory 09:25:20.22 INFO ==> Generating Spark configuration file... find: '/docker-entrypoint-initdb.d/': No such file or directory 09:25:20.23 INFO ==> No custom scripts in /docker-entrypoint-initdb.d 09:25:20.24 INFO ==> ** Spark setup finished! ** 09:25:20.26 INFO ==> ** Starting Spark in master mode ** starting org.apache.spark.deploy.master.Master, logging to /opt/bitnami/spark/logs/spark--org.apache.spark.deploy.master.Master-1-39bd998c4ce0.out Spark Command: /opt/bitnami/java/bin/java -cp /opt/bitnami/spark/conf/:/opt/bitnami/spark/jars/* -Xmx1g org.apache.spark.deploy.master.Master --host 39bd998c4ce0 --port 7077 --webui-port 8080 ======================================== Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties 24/10/12 09:25:22 INFO Master: Started daemon with process name: 66@39bd998c4ce0 24/10/12 09:25:22 INFO SignalUtils: Registering signal handler for TERM 24/10/12 09:25:22 INFO SignalUtils: Registering signal handler for HUP 24/10/12 09:25:22 INFO SignalUtils: Registering signal handler for INT 24/10/12 09:25:22 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 24/10/12 09:25:22 INFO SecurityManager: Changing view acls to: spark 24/10/12 09:25:22 INFO SecurityManager: Changing modify acls to: spark 24/10/12 09:25:22 INFO SecurityManager: Changing view acls groups to: 24/10/12 09:25:22 INFO SecurityManager: Changing modify acls groups to: 24/10/12 09:25:22 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spark); groups with view permissions: Set(); users with modify permissions: Set(spark); groups with modify permissions: Set() 24/10/12 09:25:23 INFO Utils: Successfully started service 'sparkMaster' on port 7077. 24/10/12 09:25:23 INFO Master: Starting Spark master at spark://39bd998c4ce0:7077 24/10/12 09:25:23 INFO Master: Running Spark version 3.3.0 24/10/12 09:25:23 INFO Utils: Successfully started service 'MasterUI' on port 8080. 24/10/12 09:25:23 INFO MasterWebUI: Bound MasterWebUI to 0.0.0.0, and started at http://39bd998c4ce0:8080 24/10/12 09:25:24 INFO Master: I have been elected leader! New state: ALIVE -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
