Re: Including option for starting job and task managers in the foreground

2015-12-03 Thread Maximilian Michels
I think the way supervisor is used in the Docker scripts is a bit hacky. It is simply started in the foreground and does nothing. Supervisor is actually a really nice utility to start processes in Docker containers and monitor them. Nevertheless, supervisor also expects commands to stay in the

Re: Including option for starting job and task managers in the foreground

2015-12-03 Thread Brian Chhun
Thanks Max, I took a look at making this change directly to the scripts. I was initially thinking about making a separate script whose only responsibility is to run the command in the foreground, so that the flink-daemon.sh could delegate to this script. I didn't get very far into though, mostly

Re: Including option for starting job and task managers in the foreground

2015-12-02 Thread Brian Chhun
Yep, I think this makes sense. I'm currently patching the flink-daemon.sh script to remove the `&`, but I don't think it's a very robust solution, particularly when this script changes across versions of Flink. I'm very new to Docker, but the resources I've found indicates that the process must

Re: Including option for starting job and task managers in the foreground

2015-12-02 Thread Till Rohrmann
Hi Brian, as far as I know this is at the moment not possible with our scripts. However it should be relatively easy to add by simply executing the Java command in flink-daemon.sh in the foreground. Do you want to add this? Cheers, Till On Dec 1, 2015 9:40 PM, "Brian Chhun"

Re: Including option for starting job and task managers in the foreground

2015-12-02 Thread Maximilian Michels
Hi Brian, I don't recall Docker requires commands to run in the foreground. Still, if that is your requirement, simply remove the "&" at the end of this line in flink-daemon.sh: $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" -classpath "`manglePathList

Including option for starting job and task managers in the foreground

2015-12-01 Thread Brian Chhun
Hi All, Is it possible to include a command line flag for starting job and task managers in the foreground? Currently, `bin/jobmanager.sh` and `bin/taskmanager.sh` rely on `bin/flink-daemon.sh`, which starts these things in the background. I'd like to execute these commands inside a docker