I am using my own framework, and the full task info I'm using is posted earlier in this thread. Do you happen to know if Marathon uses Mesos's health checks for its health check system?
> On Oct 6, 2015, at 9:01 PM, haosdent <[email protected]> wrote: > > Yes, launch the health task through its definition in taskinfo. Do you launch > your task through Marathon? I could test it in my side. > >> On Wed, Oct 7, 2015 at 11:56 AM, Jay Taylor <[email protected]> wrote: >> Precisely, and there are none of those statements. Are you or others >> confident health-checks are part of the code path when defined via task info >> for docker container tasks? Going through the code, I wasn't able to find >> the linkage for anything other than health-checks triggered through a custom >> executor. >> >> With that being said it is a pretty good sized code base and I'm not very >> familiar with it, so my analysis this far has by no means been exhaustive. >> >> >> >>> On Oct 6, 2015, at 8:41 PM, haosdent <[email protected]> wrote: >>> >>> When health check launch, it would have a log like this in your executor >>> stdout >>> ``` >>> Health check process launched at pid xxx >>> ``` >>> >>>> On Wed, Oct 7, 2015 at 11:37 AM, Jay Taylor <[email protected]> wrote: >>>> I'm happy to try this, however wouldn't there be output in the logs with >>>> the string "health" or "Health" if the health-check were active? None of >>>> my master or slave logs contain the string.. >>>> >>>> >>>> >>>>> On Oct 6, 2015, at 7:45 PM, haosdent <[email protected]> wrote: >>>>> >>>>> Could you use "exit 1" instead of "sleep 5" to see whether could see >>>>> unhealthy status in your task stdout/stderr. >>>>> >>>>>> On Wed, Oct 7, 2015 at 10:38 AM, Jay Taylor <[email protected]> wrote: >>>>>> My current version is 0.24.1. >>>>>> >>>>>>> On Tue, Oct 6, 2015 at 7:30 PM, haosdent <[email protected]> wrote: >>>>>>> yes, adam also help commit it to 0.23.1 and 0.24.1 >>>>>>> https://github.com/apache/mesos/commit/8c0ed92de3925d4312429bfba01b9b1ccbcbbef0 >>>>>>> >>>>>>> https://github.com/apache/mesos/commit/09e367cd69aa39c156c9326d44f4a7b829ba3db7 >>>>>>> Are you use one of this version? >>>>>>> >>>>>>>> On Wed, Oct 7, 2015 at 10:26 AM, haosdent <[email protected]> wrote: >>>>>>>> I remember 0.23.1 and 0.24.1 contains this backport, let me double >>>>>>>> check. >>>>>>>> >>>>>>>>> On Wed, Oct 7, 2015 at 10:01 AM, Jay Taylor <[email protected]> >>>>>>>>> wrote: >>>>>>>>> Oops- Now I see you already said it's in master. I'll look there :) >>>>>>>>> >>>>>>>>> Thanks again! >>>>>>>>> >>>>>>>>>> On Tue, Oct 6, 2015 at 6:59 PM, Jay Taylor <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> Great, thanks for the quick reply Tim! >>>>>>>>>> >>>>>>>>>> Do you know if there is a branch I can checkout to test it out? >>>>>>>>>> >>>>>>>>>>> On Tue, Oct 6, 2015 at 6:54 PM, Timothy Chen <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> Hi Jay, >>>>>>>>>>> >>>>>>>>>>> We just added health check support for docker tasks that's in >>>>>>>>>>> master but not yet released. It will run docker exec with the >>>>>>>>>>> command you provided as health checks. >>>>>>>>>>> >>>>>>>>>>> It should be in the next release. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>>> Tim >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Oct 6, 2015, at 6:49 PM, Jay Taylor <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Does Mesos support health checks for docker image tasks? Mesos >>>>>>>>>>>> seems to be ignoring the TaskInfo.HealthCheck field for me. >>>>>>>>>>>> >>>>>>>>>>>> Example TaskInfo JSON received back from Mesos: >>>>>>>>>>>> >>>>>>>>>>>>>> { >>>>>>>>>>>>>> "name":"hello-app.web.v3", >>>>>>>>>>>>>> "task_id":{ >>>>>>>>>>>>>> >>>>>>>>>>>>>> "value":"hello-app_web-v3.fc05a1a5-1e06-4e61-9879-be0d97cd3eec" >>>>>>>>>>>>>> }, >>>>>>>>>>>>>> "slave_id":{ >>>>>>>>>>>>>> "value":"20150924-210922-1608624320-5050-1792-S1" >>>>>>>>>>>>>> }, >>>>>>>>>>>>>> "resources":[ >>>>>>>>>>>>>> { >>>>>>>>>>>>>> "name":"cpus", >>>>>>>>>>>>>> "type":0, >>>>>>>>>>>>>> "scalar":{ >>>>>>>>>>>>>> "value":0.1 >>>>>>>>>>>>>> } >>>>>>>>>>>>>> }, >>>>>>>>>>>>>> { >>>>>>>>>>>>>> "name":"mem", >>>>>>>>>>>>>> "type":0, >>>>>>>>>>>>>> "scalar":{ >>>>>>>>>>>>>> "value":256 >>>>>>>>>>>>>> } >>>>>>>>>>>>>> }, >>>>>>>>>>>>>> { >>>>>>>>>>>>>> "name":"ports", >>>>>>>>>>>>>> "type":1, >>>>>>>>>>>>>> "ranges":{ >>>>>>>>>>>>>> "range":[ >>>>>>>>>>>>>> { >>>>>>>>>>>>>> "begin":31002, >>>>>>>>>>>>>> "end":31002 >>>>>>>>>>>>>> } >>>>>>>>>>>>>> ] >>>>>>>>>>>>>> } >>>>>>>>>>>>>> } >>>>>>>>>>>>>> ], >>>>>>>>>>>>>> "command":{ >>>>>>>>>>>>>> "container":{ >>>>>>>>>>>>>> >>>>>>>>>>>>>> "image":"docker-services1a:5000/test/app-81-1-hello-app-103" >>>>>>>>>>>>>> }, >>>>>>>>>>>>>> "shell":false >>>>>>>>>>>>>> }, >>>>>>>>>>>>>> "container":{ >>>>>>>>>>>>>> "type":1, >>>>>>>>>>>>>> "docker":{ >>>>>>>>>>>>>> >>>>>>>>>>>>>> "image":"docker-services1a:5000/gig1/app-81-1-hello-app-103", >>>>>>>>>>>>>> "network":2, >>>>>>>>>>>>>> "port_mappings":[ >>>>>>>>>>>>>> { >>>>>>>>>>>>>> "host_port":31002, >>>>>>>>>>>>>> "container_port":8000, >>>>>>>>>>>>>> "protocol":"tcp" >>>>>>>>>>>>>> } >>>>>>>>>>>>>> ], >>>>>>>>>>>>>> "privileged":false, >>>>>>>>>>>>>> "parameters":[], >>>>>>>>>>>>>> "force_pull_image":false >>>>>>>>>>>>>> } >>>>>>>>>>>>>> }, >>>>>>>>>>>>>> "health_check":{ >>>>>>>>>>>>>> "delay_seconds":5, >>>>>>>>>>>>>> "interval_seconds":10, >>>>>>>>>>>>>> "timeout_seconds":10, >>>>>>>>>>>>>> "consecutive_failures":3, >>>>>>>>>>>>>> "grace_period_seconds":0, >>>>>>>>>>>>>> "command":{ >>>>>>>>>>>>>> "shell":true, >>>>>>>>>>>>>> "value":"sleep 5", >>>>>>>>>>>>>> "user":"root" >>>>>>>>>>>>>> } >>>>>>>>>>>>>> } >>>>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> I have searched all machines and containers to see if they ever >>>>>>>>>>>> run the command (in this case `sleep 5`), but have not found any >>>>>>>>>>>> indication that it is being executed. >>>>>>>>>>>> >>>>>>>>>>>> In the mesos src code the health-checks are invoked from >>>>>>>>>>>> src/launcher/executor.cpp CommandExecutorProcess::launchTask. >>>>>>>>>>>> Does this mean that health-checks are only supported for custom >>>>>>>>>>>> executors and not for docker tasks? >>>>>>>>>>>> >>>>>>>>>>>> What I am trying to accomplish is to have the 0/non-zero >>>>>>>>>>>> exit-status of a health-check command translate to task health. >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> Jay >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Best Regards, >>>>>>>> Haosdent Huang >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Best Regards, >>>>>>> Haosdent Huang >>>>> >>>>> >>>>> >>>>> -- >>>>> Best Regards, >>>>> Haosdent Huang >>> >>> >>> >>> -- >>> Best Regards, >>> Haosdent Huang > > > > -- > Best Regards, > Haosdent Huang
