I think the way supervisor is used in the Docker scripts is a bit hacky. It is simply started in the foreground and does nothing. Supervisor is actually a really nice utility to start processes in Docker containers and monitor them.
Nevertheless, supervisor also expects commands to stay in the foreground. A common way to work around this, is to create a script which monitors the daemon process' pid. Thinking about this, I think we could actually add the foreground functionality directly in the jobmanager / taskmanager shell script like you suggested. In the meantime, you could also use a simple script like this: #!/usr/bin/env bash # daemonize job manager ./bin/jobmanager start cluster # wait until process goes down wait $! Cheers, Max On Wed, Dec 2, 2015 at 7:16 PM, Brian Chhun <brian.ch...@getbraintree.com> wrote: > Thanks, I'm basing the things I'm doing based on what I see there. One > thing that's not clear to me in that example is why supervisor is used to > keep the container alive, rather than using some simpler means. It doesn't > look like it's been configured to supervise anything. > > On Wed, Dec 2, 2015 at 11:44 AM, Maximilian Michels <m...@apache.org> > wrote: > >> Have you looked at >> https://github.com/apache/flink/tree/master/flink-contrib/docker-flink >> ? This demonstrates how to use Flink with Docker. In particular it >> states: "Images [..] run Supervisor to stay alive when running >> containers." >> >> Have a look at flink/config-flink.sh. >> >> Cheers, >> Max >> >> On Wed, Dec 2, 2015 at 6:29 PM, Brian Chhun >> <brian.ch...@getbraintree.com> wrote: >> > Yep, I think this makes sense. I'm currently patching the >> flink-daemon.sh >> > script to remove the `&`, but I don't think it's a very robust solution, >> > particularly when this script changes across versions of Flink. I'm >> very new >> > to Docker, but the resources I've found indicates that the process must >> run >> > in the foreground, though people seem to get around it with some hacks. >> > >> > When I have some time, I can look into refactoring some parts of the >> scripts >> > so that it can be started in the foreground. >> > >> > Thanks, >> > Brian >> > >> > On Wed, Dec 2, 2015 at 3:22 AM, Maximilian Michels <m...@apache.org> >> wrote: >> >> >> >> Hi Brian, >> >> >> >> I don't recall Docker requires commands to run in the foreground. >> Still, >> >> if that is your requirement, simply remove the "&" at the end of this >> line >> >> in flink-daemon.sh: >> >> >> >> $JAVA_RUN $JVM_ARGS ${FLINK_ENV_JAVA_OPTS} "${log_setting[@]}" >> -classpath >> >> "`manglePathList "$FLINK_TM_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`" >> >> ${CLASS_TO_RUN} "${ARGS[@]}" > "$out" 2>&1 < /dev/null & >> >> >> >> Cheers, >> >> Max >> >> >> >> On Wed, Dec 2, 2015 at 9:26 AM, Till Rohrmann <trohrm...@apache.org> >> >> wrote: >> >>> >> >>> Hi Brian, >> >>> >> >>> as far as I know this is at the moment not possible with our scripts. >> >>> However it should be relatively easy to add by simply executing the >> Java >> >>> command in flink-daemon.sh in the foreground. Do you want to add this? >> >>> >> >>> Cheers, >> >>> Till >> >>> >> >>> On Dec 1, 2015 9:40 PM, "Brian Chhun" <brian.ch...@getbraintree.com> >> >>> wrote: >> >>>> >> >>>> Hi All, >> >>>> >> >>>> Is it possible to include a command line flag for starting job and >> task >> >>>> managers in the foreground? Currently, `bin/jobmanager.sh` and >> >>>> `bin/taskmanager.sh` rely on `bin/flink-daemon.sh`, which starts >> these >> >>>> things in the background. I'd like to execute these commands inside >> a docker >> >>>> container, but it's expected that the process is running in the >> foreground. >> >>>> I think it might be useful to have it run in the foreground so that >> it can >> >>>> be hooked into some process supervisors. Any suggestions are >> appreciated. >> >>>> >> >>>> >> >>>> Thanks, >> >>>> Brian >> >> >> >> >> > >> > >
