Hello, I'm trying to kick off a spark streaming job to a stand alone master using spark submit inside of init.d. This is what I have:
DAEMON="spark-submit --class Streamer --executor-memory 500M --total-executor-cores 4 /path/to/assembly.jar" start() { $DAEMON -p /var/run/my_assembly.pid & echo "OK" && return 0 } However, will return 0 even if spark_submit fails. Is there a way to run spark-submit in the background and return 0 only if it successfully starts up? Or better yet, is there something in spark-submit that will allow me to do this, perhaps via a command line argument? Thanks, Ashic.