Had a discussion with Vinod and AlexR on this. I DO think it's common that frameworks do not know about the IP of the container before launching it. For instance, any networking solution that has a dynamic IPAM (e.g., calico, dc/os overlay, weave, etc.).
For those cases, probably doesn't make sense to ask the frameworks (or users) to specify the health check ip addresses for their containers. One solution we discussed was to use the container ip by default, but optionally allow people to specify `LOCALHOST` if they want to (this is what k8s does for probe. what's different is that k8s performs health check from kublet which runs outside the container network, while Mesos perform health checks from the executor which is in the same network as the container). However, the downside for this approach is that if a container joins multiple networks, what should be the address that Mesos uses to do health check by default? Therefore, I personally prefer a more consistent semantics (i.e., always using localhost). Although we can always add a backdoor for allowing people to specify a custom address for health check, but this should be avoided if possible. - Jie On Wed, Jan 18, 2017 at 10:44 AM, CmingXu <[email protected]> wrote: > The framework user have to make sure the IPs assigned unique and > accessible within the VLAN. > > In some cases, framework user want their DB, Cache or Proxy type of apps > handled by my framework & mesos, they might also want the app treated as > they are deployed as the old way, which have a unique IP for each > container. > > This kind of app is not the only type that my framework support, with > BRIDGE driver supported too. > > On Wed, Jan 18, 2017 at 5:30 PM, Jie Yu <[email protected]> wrote: > >> It's also possible that the IP is not known by the task/framework upfront >> (in fact, this is quite common depending on the underlying network driver), >> what does your general framework do in this case? >> >> - Jie >> >> On Wed, Jan 18, 2017 at 10:26 AM, CmingXu <[email protected]> wrote: >> >>> I am not sure what kind of apps are going to be running on Mesos, what I >>> am doing is a general purpose framework kind of like Marathon. >>> >>> On Wed, Jan 18, 2017 at 5:24 PM, Jie Yu <[email protected]> wrote: >>> >>>> and we don't know if the task listening on all interfaces or not >>>> >>>> >>>> OK, i think that's the reason. Although, i am wondering: the task is >>>> already listening on an external IP, why not just listen on 0.0.0.0. Any >>>> specific reason this is a concern? Or just because there is no way to >>>> configure the listening address of the task? >>>> >>>> - Jie >>>> >>>> On Wed, Jan 18, 2017 at 10:17 AM, CmingXu <[email protected]> wrote: >>>> >>>>> To Alex: >>>>> Yes, we know the IP upfront, framework user need reserve unique IP for >>>>> each task, and we don't know if the task listening on all interfaces or >>>>> not, so let the health check on the IP is the best option. >>>>> >>>>> >>>>> >>>>> To Jie Yu: >>>>> >>>>> by DEFAULT_DOMAIN I mean >>>>> >>>>> *static const string DEFAULT_DOMAIN = "127.0.0.1"* >>>>> >>>>> in source code src/health-check/health_checker.cpp >>>>> >>>>> On Wed, Jan 18, 2017 at 4:58 PM, Jie Yu <[email protected]> wrote: >>>>> >>>>>> So you want to use the IP addressed assigned by your macvlan driver >>>>>> to do health check? If that's the case, I still don't understand why >>>>>> entering the network namespace of the container and use localhost for >>>>>> health check does not work (which is what Mesos is doing). >>>>>> >>>>>> I walked through the Mesos source code and obviously the TCP & HTTP >>>>>>> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded >>>>>> >>>>>> >>>>>> What do you mean by DEFAULT_DOMAIN? >>>>>> >>>>>> - Jie >>>>>> >>>>>> On Wed, Jan 18, 2017 at 9:54 AM, CmingXu <[email protected]> wrote: >>>>>> >>>>>>> the network I am currently used is USER, and each task was assigned >>>>>>> with a unique vLAN IP with the underlaying docker driver is Macvlan. >>>>>>> I >>>>>>> want my framework user have the ability to define there own >>>>>>> HealthChecks with the IP assigned to a specific task. >>>>>>> >>>>>>> I walked through the Mesos source code and obviously the TCP & HTTP >>>>>>> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the >>>>>>> only option to be might be health check with COMMAND, but if TCP does >>>>>>> support passing IP would be great help. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu <[email protected]> wrote: >>>>>>> > Hi, can you elaborate a bit more on why you need to use an >>>>>>> customized IP, >>>>>>> > rather than using localhost for health check? >>>>>>> > >>>>>>> > - Jie >>>>>>> > >>>>>>> > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu <[email protected]> >>>>>>> wrote: >>>>>>> >> >>>>>>> >> Is there any plan we support customized IP when define a health >>>>>>> check? >>>>>>> >> If true, what's the ETA? >>>>>>> >> >>>>>>> >> thanks >>>>>>> > >>>>>>> > >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
