marathon also use mesos health check. When I use health check, I could saw the log like this in executor stdout.
``` Registered docker executor on xxxxx Starting task test-health-check.822a5fd2-6cba-11e5-b5ce-0a0027000000 Launching health check process: /home/haosdent/mesos/build/src/.libs/mesos-health-check --executor=xxxx Health check process launched at pid: 9895 Received task health update, healthy: true ``` On Wed, Oct 7, 2015 at 12:51 PM, Jay Taylor <[email protected]> wrote: > I am using my own framework, and the full task info I'm using is posted > earlier in this thread. Do you happen to know if Marathon uses Mesos's > health checks for its health check system? > > > > On Oct 6, 2015, at 9:01 PM, haosdent <[email protected]> wrote: > > Yes, launch the health task through its definition in taskinfo. Do you > launch your task through Marathon? I could test it in my side. > > On Wed, Oct 7, 2015 at 11:56 AM, Jay Taylor <[email protected]> wrote: > >> Precisely, and there are none of those statements. Are you or others >> confident health-checks are part of the code path when defined via task >> info for docker container tasks? Going through the code, I wasn't able to >> find the linkage for anything other than health-checks triggered through a >> custom executor. >> >> With that being said it is a pretty good sized code base and I'm not very >> familiar with it, so my analysis this far has by no means been exhaustive. >> >> >> >> On Oct 6, 2015, at 8:41 PM, haosdent <[email protected]> wrote: >> >> When health check launch, it would have a log like this in your executor >> stdout >> ``` >> Health check process launched at pid xxx >> ``` >> >> On Wed, Oct 7, 2015 at 11:37 AM, Jay Taylor <[email protected]> wrote: >> >>> I'm happy to try this, however wouldn't there be output in the logs with >>> the string "health" or "Health" if the health-check were active? None of >>> my master or slave logs contain the string.. >>> >>> >>> >>> On Oct 6, 2015, at 7:45 PM, haosdent <[email protected]> wrote: >>> >>> Could you use "exit 1" instead of "sleep 5" to see whether could see >>> unhealthy status in your task stdout/stderr. >>> >>> On Wed, Oct 7, 2015 at 10:38 AM, Jay Taylor <[email protected]> wrote: >>> >>>> My current version is 0.24.1. >>>> >>>> On Tue, Oct 6, 2015 at 7:30 PM, haosdent <[email protected]> wrote: >>>> >>>>> yes, adam also help commit it to 0.23.1 and 0.24.1 >>>>> https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0 >>>>> >>>>> https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7 >>>>> Are you use one of this version? >>>>> >>>>> On Wed, Oct 7, 2015 at 10:26 AM, haosdent <[email protected]> wrote: >>>>> >>>>>> I remember 0.23.1 and 0.24.1 contains this backport, let me double >>>>>> check. >>>>>> >>>>>> On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Oops- Now I see you already said it's in master. I'll look there :) >>>>>>> >>>>>>> Thanks again! >>>>>>> >>>>>>> On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Great, thanks for the quick reply Tim! >>>>>>>> >>>>>>>> Do you know if there is a branch I can checkout to test it out? >>>>>>>> >>>>>>>> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Jay, >>>>>>>>> >>>>>>>>> We just added health check support for docker tasks that's in >>>>>>>>> master but not yet released. It will run docker exec with the command >>>>>>>>> you >>>>>>>>> provided as health checks. >>>>>>>>> >>>>>>>>> It should be in the next release. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> Tim >>>>>>>>> >>>>>>>>> >>>>>>>>> On Oct 6, 2015, at 6:49 PM, Jay Taylor <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Does Mesos support health checks for docker image tasks? Mesos >>>>>>>>> seems to be ignoring the TaskInfo.HealthCheck field for me. >>>>>>>>> >>>>>>>>> Example TaskInfo JSON received back from Mesos: >>>>>>>>> >>>>>>>>> { >>>>>>>>>> >>>>>>>>>> "name":"hello-app.web.v3", >>>>>>>>>> >>>>>>>>>> "task_id":{ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec" >>>>>>>>>> >>>>>>>>>> }, >>>>>>>>>> >>>>>>>>>> "slave_id":{ >>>>>>>>>> >>>>>>>>>> "value":"20150924-210922-1608624320-5050-1792-S1" >>>>>>>>>> >>>>>>>>>> }, >>>>>>>>>> >>>>>>>>>> "resources":[ >>>>>>>>>> >>>>>>>>>> { >>>>>>>>>> >>>>>>>>>> "name":"cpus", >>>>>>>>>> >>>>>>>>>> "type":0, >>>>>>>>>> >>>>>>>>>> "scalar":{ >>>>>>>>>> >>>>>>>>>> "value":0.1 >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> }, >>>>>>>>>> >>>>>>>>>> { >>>>>>>>>> >>>>>>>>>> "name":"mem", >>>>>>>>>> >>>>>>>>>> "type":0, >>>>>>>>>> >>>>>>>>>> "scalar":{ >>>>>>>>>> >>>>>>>>>> "value":256 >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> }, >>>>>>>>>> >>>>>>>>>> { >>>>>>>>>> >>>>>>>>>> "name":"ports", >>>>>>>>>> >>>>>>>>>> "type":1, >>>>>>>>>> >>>>>>>>>> "ranges":{ >>>>>>>>>> >>>>>>>>>> "range":[ >>>>>>>>>> >>>>>>>>>> { >>>>>>>>>> >>>>>>>>>> "begin":31002, >>>>>>>>>> >>>>>>>>>> "end":31002 >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> ] >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> ], >>>>>>>>>> >>>>>>>>>> "command":{ >>>>>>>>>> >>>>>>>>>> "container":{ >>>>>>>>>> >>>>>>>>>> "image":"docker-services1a:5000/test/app-81-1-hello-app-103" >>>>>>>>>> >>>>>>>>>> }, >>>>>>>>>> >>>>>>>>>> "shell":false >>>>>>>>>> >>>>>>>>>> }, >>>>>>>>>> >>>>>>>>>> "container":{ >>>>>>>>>> >>>>>>>>>> "type":1, >>>>>>>>>> >>>>>>>>>> "docker":{ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103", >>>>>>>>>> >>>>>>>>>> "network":2, >>>>>>>>>> >>>>>>>>>> "port_mappings":[ >>>>>>>>>> >>>>>>>>>> { >>>>>>>>>> >>>>>>>>>> "host_port":31002, >>>>>>>>>> >>>>>>>>>> "container_port":8000, >>>>>>>>>> >>>>>>>>>> "protocol":"tcp" >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> ], >>>>>>>>>> >>>>>>>>>> "privileged":false, >>>>>>>>>> >>>>>>>>>> "parameters":[], >>>>>>>>>> >>>>>>>>>> "force_pull_image":false >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> }, >>>>>>>>>> >>>>>>>>>> "health_check":{ >>>>>>>>>> >>>>>>>>>> "delay_seconds":5, >>>>>>>>>> >>>>>>>>>> "interval_seconds":10, >>>>>>>>>> >>>>>>>>>> "timeout_seconds":10, >>>>>>>>>> >>>>>>>>>> "consecutive_failures":3, >>>>>>>>>> >>>>>>>>>> "grace_period_seconds":0, >>>>>>>>>> >>>>>>>>>> "command":{ >>>>>>>>>> >>>>>>>>>> "shell":true, >>>>>>>>>> >>>>>>>>>> "value":"sleep 5", >>>>>>>>>> >>>>>>>>>> "user":"root" >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> >>>>>>>>> I have searched all machines and containers to see if they ever >>>>>>>>> run the command (in this case `sleep 5`), but have not found any >>>>>>>>> indication >>>>>>>>> that it is being executed. >>>>>>>>> >>>>>>>>> In the mesos src code the health-checks are invoked from >>>>>>>>> src/launcher/executor.cpp CommandExecutorProcess::launchTask. Does >>>>>>>>> this >>>>>>>>> mean that health-checks are only supported for custom executors and >>>>>>>>> not for >>>>>>>>> docker tasks? >>>>>>>>> >>>>>>>>> What I am trying to accomplish is to have the 0/non-zero >>>>>>>>> exit-status of a health-check command translate to task health. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> Jay >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards, >>>>>> Haosdent Huang >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Best Regards, >>>>> Haosdent Huang >>>>> >>>> >>>> >>> >>> >>> -- >>> Best Regards, >>> Haosdent Huang >>> >>> >> >> >> -- >> Best Regards, >> Haosdent Huang >> >> > > > -- > Best Regards, > Haosdent Huang > > -- Best Regards, Haosdent Huang
