Re: customized IP for health check
Got it, Let user specify health checks with COMMAND(not for port but pid file maybe) seems the only option I have. Thanks a alot for the response and the hard work for Mesos. On Wed, Jan 18, 2017 at 7:37 PM, Jie Yu wrote: > Had a discussion with Vinod and AlexR on this. > > I DO think it's common that frameworks do not know about the IP of the > container before launching it. For instance, any networking solution that > has a dynamic IPAM (e.g., calico, dc/os overlay, weave, etc.). > > For those cases, probably doesn't make sense to ask the frameworks (or > users) to specify the health check ip addresses for their containers. > > One solution we discussed was to use the container ip by default, but > optionally allow people to specify `LOCALHOST` if they want to (this is > what k8s does for probe. what's different is that k8s performs health check > from kublet which runs outside the container network, while Mesos perform > health checks from the executor which is in the same network as the > container). > > However, the downside for this approach is that if a container joins > multiple networks, what should be the address that Mesos uses to do health > check by default? > > Therefore, I personally prefer a more consistent semantics (i.e., always > using localhost). Although we can always add a backdoor for allowing people > to specify a custom address for health check, but this should be avoided if > possible. > > - Jie > > On Wed, Jan 18, 2017 at 10:44 AM, CmingXu wrote: > >> The framework user have to make sure the IPs assigned unique and >> accessible within the VLAN. >> >> In some cases, framework user want their DB, Cache or Proxy type of apps >> handled by my framework & mesos, they might also want the app treated as >> they are deployed as the old way, which have a unique IP for each >> container. >> >> This kind of app is not the only type that my framework support, with >> BRIDGE driver supported too. >> >> On Wed, Jan 18, 2017 at 5:30 PM, Jie Yu wrote: >> >>> It's also possible that the IP is not known by the task/framework >>> upfront (in fact, this is quite common depending on the underlying network >>> driver), what does your general framework do in this case? >>> >>> - Jie >>> >>> On Wed, Jan 18, 2017 at 10:26 AM, CmingXu wrote: >>> I am not sure what kind of apps are going to be running on Mesos, what I am doing is a general purpose framework kind of like Marathon. On Wed, Jan 18, 2017 at 5:24 PM, Jie Yu wrote: > and we don't know if the task listening on all interfaces or not > > > OK, i think that's the reason. Although, i am wondering: the task is > already listening on an external IP, why not just listen on 0.0.0.0. Any > specific reason this is a concern? Or just because there is no way to > configure the listening address of the task? > > - Jie > > On Wed, Jan 18, 2017 at 10:17 AM, CmingXu wrote: > >> To Alex: >> Yes, we know the IP upfront, framework user need reserve unique IP >> for each task, and we don't know if the task listening on all interfaces >> or >> not, so let the health check on the IP is the best option. >> >> >> >> To Jie Yu: >> >> by DEFAULT_DOMAIN I mean >> >> *static const string DEFAULT_DOMAIN = "127.0.0.1"* >> >> in source code src/health-check/health_checker.cpp >> >> On Wed, Jan 18, 2017 at 4:58 PM, Jie Yu wrote: >> >>> So you want to use the IP addressed assigned by your macvlan driver >>> to do health check? If that's the case, I still don't understand why >>> entering the network namespace of the container and use localhost for >>> health check does not work (which is what Mesos is doing). >>> >>> I walked through the Mesos source code and obviously the TCP & HTTP doesn't meet my requirements as DEFAULT_DOMAIN is hard coded >>> >>> >>> What do you mean by DEFAULT_DOMAIN? >>> >>> - Jie >>> >>> On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: >>> the network I am currently used is USER, and each task was assigned with a unique vLAN IP with the underlaying docker driver is Macvlan. I want my framework user have the ability to define there own HealthChecks with the IP assigned to a specific task. I walked through the Mesos source code and obviously the TCP & HTTP doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the only option to be might be health check with COMMAND, but if TCP does support passing IP would be great help. Thanks On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: > Hi, can you elaborate a bit more on why you need to use an customized IP, > rather than using localhost for health check? > > - Jie > > On
Re: customized IP for health check
Had a discussion with Vinod and AlexR on this. I DO think it's common that frameworks do not know about the IP of the container before launching it. For instance, any networking solution that has a dynamic IPAM (e.g., calico, dc/os overlay, weave, etc.). For those cases, probably doesn't make sense to ask the frameworks (or users) to specify the health check ip addresses for their containers. One solution we discussed was to use the container ip by default, but optionally allow people to specify `LOCALHOST` if they want to (this is what k8s does for probe. what's different is that k8s performs health check from kublet which runs outside the container network, while Mesos perform health checks from the executor which is in the same network as the container). However, the downside for this approach is that if a container joins multiple networks, what should be the address that Mesos uses to do health check by default? Therefore, I personally prefer a more consistent semantics (i.e., always using localhost). Although we can always add a backdoor for allowing people to specify a custom address for health check, but this should be avoided if possible. - Jie On Wed, Jan 18, 2017 at 10:44 AM, CmingXu wrote: > The framework user have to make sure the IPs assigned unique and > accessible within the VLAN. > > In some cases, framework user want their DB, Cache or Proxy type of apps > handled by my framework & mesos, they might also want the app treated as > they are deployed as the old way, which have a unique IP for each > container. > > This kind of app is not the only type that my framework support, with > BRIDGE driver supported too. > > On Wed, Jan 18, 2017 at 5:30 PM, Jie Yu wrote: > >> It's also possible that the IP is not known by the task/framework upfront >> (in fact, this is quite common depending on the underlying network driver), >> what does your general framework do in this case? >> >> - Jie >> >> On Wed, Jan 18, 2017 at 10:26 AM, CmingXu wrote: >> >>> I am not sure what kind of apps are going to be running on Mesos, what I >>> am doing is a general purpose framework kind of like Marathon. >>> >>> On Wed, Jan 18, 2017 at 5:24 PM, Jie Yu wrote: >>> and we don't know if the task listening on all interfaces or not OK, i think that's the reason. Although, i am wondering: the task is already listening on an external IP, why not just listen on 0.0.0.0. Any specific reason this is a concern? Or just because there is no way to configure the listening address of the task? - Jie On Wed, Jan 18, 2017 at 10:17 AM, CmingXu wrote: > To Alex: > Yes, we know the IP upfront, framework user need reserve unique IP for > each task, and we don't know if the task listening on all interfaces or > not, so let the health check on the IP is the best option. > > > > To Jie Yu: > > by DEFAULT_DOMAIN I mean > > *static const string DEFAULT_DOMAIN = "127.0.0.1"* > > in source code src/health-check/health_checker.cpp > > On Wed, Jan 18, 2017 at 4:58 PM, Jie Yu wrote: > >> So you want to use the IP addressed assigned by your macvlan driver >> to do health check? If that's the case, I still don't understand why >> entering the network namespace of the container and use localhost for >> health check does not work (which is what Mesos is doing). >> >> I walked through the Mesos source code and obviously the TCP & HTTP >>> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded >> >> >> What do you mean by DEFAULT_DOMAIN? >> >> - Jie >> >> On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: >> >>> the network I am currently used is USER, and each task was assigned >>> with a unique vLAN IP with the underlaying docker driver is Macvlan. >>> I >>> want my framework user have the ability to define there own >>> HealthChecks with the IP assigned to a specific task. >>> >>> I walked through the Mesos source code and obviously the TCP & HTTP >>> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the >>> only option to be might be health check with COMMAND, but if TCP does >>> support passing IP would be great help. >>> >>> Thanks >>> >>> On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: >>> > Hi, can you elaborate a bit more on why you need to use an >>> customized IP, >>> > rather than using localhost for health check? >>> > >>> > - Jie >>> > >>> > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu >>> wrote: >>> >> >>> >> Is there any plan we support customized IP when define a health >>> check? >>> >> If true, what's the ETA? >>> >> >>> >> thanks >>> > >>> > >>> >> >> > >>> >> >
Re: customized IP for health check
The framework user have to make sure the IPs assigned unique and accessible within the VLAN. In some cases, framework user want their DB, Cache or Proxy type of apps handled by my framework & mesos, they might also want the app treated as they are deployed as the old way, which have a unique IP for each container. This kind of app is not the only type that my framework support, with BRIDGE driver supported too. On Wed, Jan 18, 2017 at 5:30 PM, Jie Yu wrote: > It's also possible that the IP is not known by the task/framework upfront > (in fact, this is quite common depending on the underlying network driver), > what does your general framework do in this case? > > - Jie > > On Wed, Jan 18, 2017 at 10:26 AM, CmingXu wrote: > >> I am not sure what kind of apps are going to be running on Mesos, what I >> am doing is a general purpose framework kind of like Marathon. >> >> On Wed, Jan 18, 2017 at 5:24 PM, Jie Yu wrote: >> >>> and we don't know if the task listening on all interfaces or not >>> >>> >>> OK, i think that's the reason. Although, i am wondering: the task is >>> already listening on an external IP, why not just listen on 0.0.0.0. Any >>> specific reason this is a concern? Or just because there is no way to >>> configure the listening address of the task? >>> >>> - Jie >>> >>> On Wed, Jan 18, 2017 at 10:17 AM, CmingXu wrote: >>> To Alex: Yes, we know the IP upfront, framework user need reserve unique IP for each task, and we don't know if the task listening on all interfaces or not, so let the health check on the IP is the best option. To Jie Yu: by DEFAULT_DOMAIN I mean *static const string DEFAULT_DOMAIN = "127.0.0.1"* in source code src/health-check/health_checker.cpp On Wed, Jan 18, 2017 at 4:58 PM, Jie Yu wrote: > So you want to use the IP addressed assigned by your macvlan driver to > do health check? If that's the case, I still don't understand why entering > the network namespace of the container and use localhost for health check > does not work (which is what Mesos is doing). > > I walked through the Mesos source code and obviously the TCP & HTTP >> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded > > > What do you mean by DEFAULT_DOMAIN? > > - Jie > > On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: > >> the network I am currently used is USER, and each task was assigned >> with a unique vLAN IP with the underlaying docker driver is Macvlan. I >> want my framework user have the ability to define there own >> HealthChecks with the IP assigned to a specific task. >> >> I walked through the Mesos source code and obviously the TCP & HTTP >> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the >> only option to be might be health check with COMMAND, but if TCP does >> support passing IP would be great help. >> >> Thanks >> >> On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: >> > Hi, can you elaborate a bit more on why you need to use an >> customized IP, >> > rather than using localhost for health check? >> > >> > - Jie >> > >> > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu >> wrote: >> >> >> >> Is there any plan we support customized IP when define a health >> check? >> >> If true, what's the ETA? >> >> >> >> thanks >> > >> > >> > > >>> >> >
Re: customized IP for health check
It's also possible that the IP is not known by the task/framework upfront (in fact, this is quite common depending on the underlying network driver), what does your general framework do in this case? - Jie On Wed, Jan 18, 2017 at 10:26 AM, CmingXu wrote: > I am not sure what kind of apps are going to be running on Mesos, what I > am doing is a general purpose framework kind of like Marathon. > > On Wed, Jan 18, 2017 at 5:24 PM, Jie Yu wrote: > >> and we don't know if the task listening on all interfaces or not >> >> >> OK, i think that's the reason. Although, i am wondering: the task is >> already listening on an external IP, why not just listen on 0.0.0.0. Any >> specific reason this is a concern? Or just because there is no way to >> configure the listening address of the task? >> >> - Jie >> >> On Wed, Jan 18, 2017 at 10:17 AM, CmingXu wrote: >> >>> To Alex: >>> Yes, we know the IP upfront, framework user need reserve unique IP for >>> each task, and we don't know if the task listening on all interfaces or >>> not, so let the health check on the IP is the best option. >>> >>> >>> >>> To Jie Yu: >>> >>> by DEFAULT_DOMAIN I mean >>> >>> *static const string DEFAULT_DOMAIN = "127.0.0.1"* >>> >>> in source code src/health-check/health_checker.cpp >>> >>> On Wed, Jan 18, 2017 at 4:58 PM, Jie Yu wrote: >>> So you want to use the IP addressed assigned by your macvlan driver to do health check? If that's the case, I still don't understand why entering the network namespace of the container and use localhost for health check does not work (which is what Mesos is doing). I walked through the Mesos source code and obviously the TCP & HTTP > doesn't meet my requirements as DEFAULT_DOMAIN is hard coded What do you mean by DEFAULT_DOMAIN? - Jie On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: > the network I am currently used is USER, and each task was assigned > with a unique vLAN IP with the underlaying docker driver is Macvlan. I > want my framework user have the ability to define there own > HealthChecks with the IP assigned to a specific task. > > I walked through the Mesos source code and obviously the TCP & HTTP > doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the > only option to be might be health check with COMMAND, but if TCP does > support passing IP would be great help. > > Thanks > > On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: > > Hi, can you elaborate a bit more on why you need to use an > customized IP, > > rather than using localhost for health check? > > > > - Jie > > > > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu wrote: > >> > >> Is there any plan we support customized IP when define a health > check? > >> If true, what's the ETA? > >> > >> thanks > > > > > >>> >> >
Re: customized IP for health check
I am not sure what kind of apps are going to be running on Mesos, what I am doing is a general purpose framework kind of like Marathon. On Wed, Jan 18, 2017 at 5:24 PM, Jie Yu wrote: > and we don't know if the task listening on all interfaces or not > > > OK, i think that's the reason. Although, i am wondering: the task is > already listening on an external IP, why not just listen on 0.0.0.0. Any > specific reason this is a concern? Or just because there is no way to > configure the listening address of the task? > > - Jie > > On Wed, Jan 18, 2017 at 10:17 AM, CmingXu wrote: > >> To Alex: >> Yes, we know the IP upfront, framework user need reserve unique IP for >> each task, and we don't know if the task listening on all interfaces or >> not, so let the health check on the IP is the best option. >> >> >> >> To Jie Yu: >> >> by DEFAULT_DOMAIN I mean >> >> *static const string DEFAULT_DOMAIN = "127.0.0.1"* >> >> in source code src/health-check/health_checker.cpp >> >> On Wed, Jan 18, 2017 at 4:58 PM, Jie Yu wrote: >> >>> So you want to use the IP addressed assigned by your macvlan driver to >>> do health check? If that's the case, I still don't understand why entering >>> the network namespace of the container and use localhost for health check >>> does not work (which is what Mesos is doing). >>> >>> I walked through the Mesos source code and obviously the TCP & HTTP doesn't meet my requirements as DEFAULT_DOMAIN is hard coded >>> >>> >>> What do you mean by DEFAULT_DOMAIN? >>> >>> - Jie >>> >>> On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: >>> the network I am currently used is USER, and each task was assigned with a unique vLAN IP with the underlaying docker driver is Macvlan. I want my framework user have the ability to define there own HealthChecks with the IP assigned to a specific task. I walked through the Mesos source code and obviously the TCP & HTTP doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the only option to be might be health check with COMMAND, but if TCP does support passing IP would be great help. Thanks On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: > Hi, can you elaborate a bit more on why you need to use an customized IP, > rather than using localhost for health check? > > - Jie > > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu wrote: >> >> Is there any plan we support customized IP when define a health check? >> If true, what's the ETA? >> >> thanks > > >>> >>> >> >
Re: customized IP for health check
> > and we don't know if the task listening on all interfaces or not OK, i think that's the reason. Although, i am wondering: the task is already listening on an external IP, why not just listen on 0.0.0.0. Any specific reason this is a concern? Or just because there is no way to configure the listening address of the task? - Jie On Wed, Jan 18, 2017 at 10:17 AM, CmingXu wrote: > To Alex: > Yes, we know the IP upfront, framework user need reserve unique IP for > each task, and we don't know if the task listening on all interfaces or > not, so let the health check on the IP is the best option. > > > > To Jie Yu: > > by DEFAULT_DOMAIN I mean > > *static const string DEFAULT_DOMAIN = "127.0.0.1"* > > in source code src/health-check/health_checker.cpp > > On Wed, Jan 18, 2017 at 4:58 PM, Jie Yu wrote: > >> So you want to use the IP addressed assigned by your macvlan driver to do >> health check? If that's the case, I still don't understand why entering the >> network namespace of the container and use localhost for health check does >> not work (which is what Mesos is doing). >> >> I walked through the Mesos source code and obviously the TCP & HTTP >>> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded >> >> >> What do you mean by DEFAULT_DOMAIN? >> >> - Jie >> >> On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: >> >>> the network I am currently used is USER, and each task was assigned >>> with a unique vLAN IP with the underlaying docker driver is Macvlan. I >>> want my framework user have the ability to define there own >>> HealthChecks with the IP assigned to a specific task. >>> >>> I walked through the Mesos source code and obviously the TCP & HTTP >>> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the >>> only option to be might be health check with COMMAND, but if TCP does >>> support passing IP would be great help. >>> >>> Thanks >>> >>> On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: >>> > Hi, can you elaborate a bit more on why you need to use an customized >>> IP, >>> > rather than using localhost for health check? >>> > >>> > - Jie >>> > >>> > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu wrote: >>> >> >>> >> Is there any plan we support customized IP when define a health check? >>> >> If true, what's the ETA? >>> >> >>> >> thanks >>> > >>> > >>> >> >> >
Re: customized IP for health check
To Alex: Yes, we know the IP upfront, framework user need reserve unique IP for each task, and we don't know if the task listening on all interfaces or not, so let the health check on the IP is the best option. To Jie Yu: by DEFAULT_DOMAIN I mean *static const string DEFAULT_DOMAIN = "127.0.0.1"* in source code src/health-check/health_checker.cpp On Wed, Jan 18, 2017 at 4:58 PM, Jie Yu wrote: > So you want to use the IP addressed assigned by your macvlan driver to do > health check? If that's the case, I still don't understand why entering the > network namespace of the container and use localhost for health check does > not work (which is what Mesos is doing). > > I walked through the Mesos source code and obviously the TCP & HTTP >> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded > > > What do you mean by DEFAULT_DOMAIN? > > - Jie > > On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: > >> the network I am currently used is USER, and each task was assigned >> with a unique vLAN IP with the underlaying docker driver is Macvlan. I >> want my framework user have the ability to define there own >> HealthChecks with the IP assigned to a specific task. >> >> I walked through the Mesos source code and obviously the TCP & HTTP >> doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the >> only option to be might be health check with COMMAND, but if TCP does >> support passing IP would be great help. >> >> Thanks >> >> On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: >> > Hi, can you elaborate a bit more on why you need to use an customized >> IP, >> > rather than using localhost for health check? >> > >> > - Jie >> > >> > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu wrote: >> >> >> >> Is there any plan we support customized IP when define a health check? >> >> If true, what's the ETA? >> >> >> >> thanks >> > >> > >> > >
Re: customized IP for health check
So you want to use the IP addressed assigned by your macvlan driver to do health check? If that's the case, I still don't understand why entering the network namespace of the container and use localhost for health check does not work (which is what Mesos is doing). I walked through the Mesos source code and obviously the TCP & HTTP > doesn't meet my requirements as DEFAULT_DOMAIN is hard coded What do you mean by DEFAULT_DOMAIN? - Jie On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: > the network I am currently used is USER, and each task was assigned > with a unique vLAN IP with the underlaying docker driver is Macvlan. I > want my framework user have the ability to define there own > HealthChecks with the IP assigned to a specific task. > > I walked through the Mesos source code and obviously the TCP & HTTP > doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the > only option to be might be health check with COMMAND, but if TCP does > support passing IP would be great help. > > Thanks > > On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: > > Hi, can you elaborate a bit more on why you need to use an customized IP, > > rather than using localhost for health check? > > > > - Jie > > > > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu wrote: > >> > >> Is there any plan we support customized IP when define a health check? > >> If true, what's the ETA? > >> > >> thanks > > > > >
Re: customized IP for health check
I'm not sure that exposing a domain will help: do you know the IP of your task upfront, i.e., at the moment when you construct TaskInfo? Isn't your task listening on all interfaces? On Wed, Jan 18, 2017 at 9:54 AM, CmingXu wrote: > the network I am currently used is USER, and each task was assigned > with a unique vLAN IP with the underlaying docker driver is Macvlan. I > want my framework user have the ability to define there own > HealthChecks with the IP assigned to a specific task. > > I walked through the Mesos source code and obviously the TCP & HTTP > doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the > only option to be might be health check with COMMAND, but if TCP does > support passing IP would be great help. > > Thanks > > On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: > > Hi, can you elaborate a bit more on why you need to use an customized IP, > > rather than using localhost for health check? > > > > - Jie > > > > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu wrote: > >> > >> Is there any plan we support customized IP when define a health check? > >> If true, what's the ETA? > >> > >> thanks > > > > >
Re: customized IP for health check
the network I am currently used is USER, and each task was assigned with a unique vLAN IP with the underlaying docker driver is Macvlan. I want my framework user have the ability to define there own HealthChecks with the IP assigned to a specific task. I walked through the Mesos source code and obviously the TCP & HTTP doesn't meet my requirements as DEFAULT_DOMAIN is hard coded, now the only option to be might be health check with COMMAND, but if TCP does support passing IP would be great help. Thanks On Wed, Jan 18, 2017 at 4:40 PM, Jie Yu wrote: > Hi, can you elaborate a bit more on why you need to use an customized IP, > rather than using localhost for health check? > > - Jie > > On Wed, Jan 18, 2017 at 9:19 AM, CmingXu wrote: >> >> Is there any plan we support customized IP when define a health check? >> If true, what's the ETA? >> >> thanks > >
Re: customized IP for health check
Hi, can you elaborate a bit more on why you need to use an customized IP, rather than using localhost for health check? - Jie On Wed, Jan 18, 2017 at 9:19 AM, CmingXu wrote: > Is there any plan we support customized IP when define a health check? > If true, what's the ETA? > > thanks >
customized IP for health check
Is there any plan we support customized IP when define a health check? If true, what's the ETA? thanks