Re: Zookeeper integration for Mesos-DNS
Aaron, It depends on what you mean however, Mesos-DNS works outside the cluster IMO. It is a bridge for things in the cluster (services launched by mesos)... But at that point it is DNS. Any client in or out of the cluster that can query DNS that leverage the service. Sent from my iPhone On Mar 23, 2015, at 4:25 AM, Aaron Carey aca...@ilm.com wrote: Hey, I don't suppose there is anything like Mesos-DNS but for services/users outside the mesos cluster? So having a service which updates a DNS provider with task port/ips running inside the cluster so that external users are able to find those services? Am I correct in thinking Mesos-DNS only works inside the cluster? Currently we're using consul for this, but I'd be interested if there was some sort of magical plug and play solution? Thanks, Aaron From: Christos Kozyrakis [kozyr...@gmail.com] Sent: 21 March 2015 00:18 To: user@mesos.apache.org Subject: Zookeeper integration for Mesos-DNS Hi everybody, we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of providing Mesos-DNS with a list of masters, you point it to the Zookeeper instances. Meson-DNS will watch Zookeeper to detect the current leading master. So, while the list of Zookeeper instances is configured in a static manner, Mesos masters can be added or removed freely without restarting Mesos-DNS. The integration with Zookeeper forced to switch from -v and -vv as the flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2 (very verbose). To reduce complications because of dependencies to other packages, we have also started using godep. Please take a look at the branch https://github.com/mesosphere/mesos-dns/tree/zk and provide us with any feedback on the code or the documentation. Thanks -- Christos
Re: Zookeeper integration for Mesos-DNS
roger that On Mar 23, 2015, at 9:22 AM, Aaron Carey aca...@ilm.com wrote: Thanks Ken, So basically we just need to add mesos-dns to our /etc/resolv.conf on every machine and hey presto auto-service discovery (using DNS)? (Here I mean service discovery to be: hey where is rabbitmq? DNS says: 172.20.121.292:8393 or whatever) Aaron From: Ken Sipe [kens...@gmail.com] Sent: 23 March 2015 14:29 To: user@mesos.apache.org Subject: Re: Zookeeper integration for Mesos-DNS Aaron, Mesos-DNS is a DNS name server + a monitor of mesos-masters. It listens to the mesos-master. If a service is launched by mesos then mesos-dns conjures a service name (app_id + framework_id +.mesos) and associates it to the IP and PORT of the service. Since Mesos-DNS is a name service, it needs to be in your list of name services for service discovery. From a service discovery stand point there is no need to be in the cluster and there is no need to have a dependency on Mesos. Mesos-DNS is not a proxy. It doesn’t provide any special services to clients or services inside the cluster. more detail below. On Mar 23, 2015, at 7:52 AM, Aaron Carey aca...@ilm.com mailto:aca...@ilm.com wrote: As I understood it, it provides a service for containers within the cluster to automatically find each other as it handles their dns calls? The way this is stated this doesn’t seem true.Mesos-DNS is a DNS name server.From a service discovery stand point, It doesn’t do anything different than a standard DNS naming server. However clients outside the cluster will not use the mesos-dns service by default, so won't have knowledge of anything running inside the cluster? This is all dependent on how /etc/resolv.conf is setup. If mesos-dns is in the list… then this is not true. Is there an easy way to set this up to (for example) add records to AWS Route 53 when services get started in the cluster, so other clients can see them? This is outside of Mesos-DNS Good Luck!! Thanks! Aaron From: Ken Sipe [kens...@gmail.com mailto:kens...@gmail.com] Sent: 23 March 2015 13:31 To: user@mesos.apache.org mailto:user@mesos.apache.org Subject: Re: Zookeeper integration for Mesos-DNS Aaron, It depends on what you mean however, Mesos-DNS works outside the cluster IMO. It is a bridge for things in the cluster (services launched by mesos)... But at that point it is DNS. Any client in or out of the cluster that can query DNS that leverage the service. Sent from my iPhone On Mar 23, 2015, at 4:25 AM, Aaron Carey aca...@ilm.com mailto:aca...@ilm.com wrote: Hey, I don't suppose there is anything like Mesos-DNS but for services/users outside the mesos cluster? So having a service which updates a DNS provider with task port/ips running inside the cluster so that external users are able to find those services? Am I correct in thinking Mesos-DNS only works inside the cluster? Currently we're using consul for this, but I'd be interested if there was some sort of magical plug and play solution? Thanks, Aaron From: Christos Kozyrakis [kozyr...@gmail.com mailto:kozyr...@gmail.com] Sent: 21 March 2015 00:18 To: user@mesos.apache.org mailto:user@mesos.apache.org Subject: Zookeeper integration for Mesos-DNS Hi everybody, we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of providing Mesos-DNS with a list of masters, you point it to the Zookeeper instances. Meson-DNS will watch Zookeeper to detect the current leading master. So, while the list of Zookeeper instances is configured in a static manner, Mesos masters can be added or removed freely without restarting Mesos-DNS. The integration with Zookeeper forced to switch from -v and -vv as the flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2 (very verbose). To reduce complications because of dependencies to other packages, we have also started using godep. Please take a look at the branch https://github.com/mesosphere/mesos-dns/tree/zk https://github.com/mesosphere/mesos-dns/tree/zk and provide us with any feedback on the code or the documentation. Thanks -- Christos
Re: Zookeeper integration for Mesos-DNS
Aaron, Mesos-DNS is a DNS name server + a monitor of mesos-masters. It listens to the mesos-master. If a service is launched by mesos then mesos-dns conjures a service name (app_id + framework_id +.mesos) and associates it to the IP and PORT of the service. Since Mesos-DNS is a name service, it needs to be in your list of name services for service discovery. From a service discovery stand point there is no need to be in the cluster and there is no need to have a dependency on Mesos. Mesos-DNS is not a proxy. It doesn’t provide any special services to clients or services inside the cluster. more detail below. On Mar 23, 2015, at 7:52 AM, Aaron Carey aca...@ilm.com wrote: As I understood it, it provides a service for containers within the cluster to automatically find each other as it handles their dns calls? The way this is stated this doesn’t seem true.Mesos-DNS is a DNS name server.From a service discovery stand point, It doesn’t do anything different than a standard DNS naming server. However clients outside the cluster will not use the mesos-dns service by default, so won't have knowledge of anything running inside the cluster? This is all dependent on how /etc/resolv.conf is setup. If mesos-dns is in the list… then this is not true. Is there an easy way to set this up to (for example) add records to AWS Route 53 when services get started in the cluster, so other clients can see them? This is outside of Mesos-DNS Good Luck!! Thanks! Aaron From: Ken Sipe [kens...@gmail.com mailto:kens...@gmail.com] Sent: 23 March 2015 13:31 To: user@mesos.apache.org mailto:user@mesos.apache.org Subject: Re: Zookeeper integration for Mesos-DNS Aaron, It depends on what you mean however, Mesos-DNS works outside the cluster IMO. It is a bridge for things in the cluster (services launched by mesos)... But at that point it is DNS. Any client in or out of the cluster that can query DNS that leverage the service. Sent from my iPhone On Mar 23, 2015, at 4:25 AM, Aaron Carey aca...@ilm.com mailto:aca...@ilm.com wrote: Hey, I don't suppose there is anything like Mesos-DNS but for services/users outside the mesos cluster? So having a service which updates a DNS provider with task port/ips running inside the cluster so that external users are able to find those services? Am I correct in thinking Mesos-DNS only works inside the cluster? Currently we're using consul for this, but I'd be interested if there was some sort of magical plug and play solution? Thanks, Aaron From: Christos Kozyrakis [kozyr...@gmail.com mailto:kozyr...@gmail.com] Sent: 21 March 2015 00:18 To: user@mesos.apache.org mailto:user@mesos.apache.org Subject: Zookeeper integration for Mesos-DNS Hi everybody, we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of providing Mesos-DNS with a list of masters, you point it to the Zookeeper instances. Meson-DNS will watch Zookeeper to detect the current leading master. So, while the list of Zookeeper instances is configured in a static manner, Mesos masters can be added or removed freely without restarting Mesos-DNS. The integration with Zookeeper forced to switch from -v and -vv as the flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2 (very verbose). To reduce complications because of dependencies to other packages, we have also started using godep. Please take a look at the branch https://github.com/mesosphere/mesos-dns/tree/zk https://github.com/mesosphere/mesos-dns/tree/zk and provide us with any feedback on the code or the documentation. Thanks -- Christos
RE: Zookeeper integration for Mesos-DNS
As I understood it, it provides a service for containers within the cluster to automatically find each other as it handles their dns calls? However clients outside the cluster will not use the mesos-dns service by default, so won't have knowledge of anything running inside the cluster? Is there an easy way to set this up to (for example) add records to AWS Route 53 when services get started in the cluster, so other clients can see them? Thanks! Aaron From: Ken Sipe [kens...@gmail.com] Sent: 23 March 2015 13:31 To: user@mesos.apache.org Subject: Re: Zookeeper integration for Mesos-DNS Aaron, It depends on what you mean however, Mesos-DNS works outside the cluster IMO. It is a bridge for things in the cluster (services launched by mesos)... But at that point it is DNS. Any client in or out of the cluster that can query DNS that leverage the service. Sent from my iPhone On Mar 23, 2015, at 4:25 AM, Aaron Carey aca...@ilm.commailto:aca...@ilm.com wrote: Hey, I don't suppose there is anything like Mesos-DNS but for services/users outside the mesos cluster? So having a service which updates a DNS provider with task port/ips running inside the cluster so that external users are able to find those services? Am I correct in thinking Mesos-DNS only works inside the cluster? Currently we're using consul for this, but I'd be interested if there was some sort of magical plug and play solution? Thanks, Aaron From: Christos Kozyrakis [kozyr...@gmail.commailto:kozyr...@gmail.com] Sent: 21 March 2015 00:18 To: user@mesos.apache.orgmailto:user@mesos.apache.org Subject: Zookeeper integration for Mesos-DNS Hi everybody, we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of providing Mesos-DNS with a list of masters, you point it to the Zookeeper instances. Meson-DNS will watch Zookeeper to detect the current leading master. So, while the list of Zookeeper instances is configured in a static manner, Mesos masters can be added or removed freely without restarting Mesos-DNS. The integration with Zookeeper forced to switch from -v and -vv as the flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2 (very verbose). To reduce complications because of dependencies to other packages, we have also started using godep. Please take a look at the branch https://github.com/mesosphere/mesos-dns/tree/zk and provide us with any feedback on the code or the documentation. Thanks -- Christos
RE: Zookeeper integration for Mesos-DNS
lovely, thanks! From: craig w [codecr...@gmail.com] Sent: 23 March 2015 15:35 To: user@mesos.apache.org Subject: Re: Zookeeper integration for Mesos-DNS Keep in mind DNS will give you the ipaddress of the host, so rabbitmq.marathon.mesos will resolve to some IP address. Do get port information you have to query mesos-dns for its SRV records. On Mon, Mar 23, 2015 at 11:29 AM, Ken Sipe kens...@gmail.commailto:kens...@gmail.com wrote: roger that On Mar 23, 2015, at 9:22 AM, Aaron Carey aca...@ilm.commailto:aca...@ilm.com wrote: Thanks Ken, So basically we just need to add mesos-dns to our /etc/resolv.conf on every machine and hey presto auto-service discovery (using DNS)? (Here I mean service discovery to be: hey where is rabbitmq? DNS says: 172.20.121.292:8393 or whatever) Aaron From: Ken Sipe [kens...@gmail.commailto:kens...@gmail.com] Sent: 23 March 2015 14:29 To: user@mesos.apache.orgmailto:user@mesos.apache.org Subject: Re: Zookeeper integration for Mesos-DNS Aaron, Mesos-DNS is a DNS name server + a monitor of mesos-masters. It listens to the mesos-master. If a service is launched by mesos then mesos-dns conjures a service name (app_id + framework_id +.mesos) and associates it to the IP and PORT of the service. Since Mesos-DNS is a name service, it needs to be in your list of name services for service discovery. From a service discovery stand point there is no need to be in the cluster and there is no need to have a dependency on Mesos. Mesos-DNS is not a proxy. It doesn’t provide any special services to clients or services inside the cluster. more detail below. On Mar 23, 2015, at 7:52 AM, Aaron Carey aca...@ilm.commailto:aca...@ilm.com wrote: As I understood it, it provides a service for containers within the cluster to automatically find each other as it handles their dns calls? The way this is stated this doesn’t seem true.Mesos-DNS is a DNS name server.From a service discovery stand point, It doesn’t do anything different than a standard DNS naming server. However clients outside the cluster will not use the mesos-dns service by default, so won't have knowledge of anything running inside the cluster? This is all dependent on how /etc/resolv.conf is setup. If mesos-dns is in the list… then this is not true. Is there an easy way to set this up to (for example) add records to AWS Route 53 when services get started in the cluster, so other clients can see them? This is outside of Mesos-DNS Good Luck!! Thanks! Aaron From: Ken Sipe [kens...@gmail.commailto:kens...@gmail.com] Sent: 23 March 2015 13:31 To: user@mesos.apache.orgmailto:user@mesos.apache.org Subject: Re: Zookeeper integration for Mesos-DNS Aaron, It depends on what you mean however, Mesos-DNS works outside the cluster IMO. It is a bridge for things in the cluster (services launched by mesos)... But at that point it is DNS. Any client in or out of the cluster that can query DNS that leverage the service. Sent from my iPhone On Mar 23, 2015, at 4:25 AM, Aaron Carey aca...@ilm.commailto:aca...@ilm.com wrote: Hey, I don't suppose there is anything like Mesos-DNS but for services/users outside the mesos cluster? So having a service which updates a DNS provider with task port/ips running inside the cluster so that external users are able to find those services? Am I correct in thinking Mesos-DNS only works inside the cluster? Currently we're using consul for this, but I'd be interested if there was some sort of magical plug and play solution? Thanks, Aaron From: Christos Kozyrakis [kozyr...@gmail.commailto:kozyr...@gmail.com] Sent: 21 March 2015 00:18 To: user@mesos.apache.orgmailto:user@mesos.apache.org Subject: Zookeeper integration for Mesos-DNS Hi everybody, we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of providing Mesos-DNS with a list of masters, you point it to the Zookeeper instances. Meson-DNS will watch Zookeeper to detect the current leading master. So, while the list of Zookeeper instances is configured in a static manner, Mesos masters can be added or removed freely without restarting Mesos-DNS. The integration with Zookeeper forced to switch from -v and -vv as the flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2 (very verbose). To reduce complications because of dependencies to other packages, we have also started using godep. Please take a look at the branch https://github.com/mesosphere/mesos-dns/tree/zk and provide us with any feedback on the code or the documentation. Thanks -- Christos -- https://github.com/mindscratch https://www.google.com/+CraigWickesser https://twitter.com/mind_scratch https://twitter.com/craig_links
RE: Zookeeper integration for Mesos-DNS
Hey, I don't suppose there is anything like Mesos-DNS but for services/users outside the mesos cluster? So having a service which updates a DNS provider with task port/ips running inside the cluster so that external users are able to find those services? Am I correct in thinking Mesos-DNS only works inside the cluster? Currently we're using consul for this, but I'd be interested if there was some sort of magical plug and play solution? Thanks, Aaron From: Christos Kozyrakis [kozyr...@gmail.com] Sent: 21 March 2015 00:18 To: user@mesos.apache.org Subject: Zookeeper integration for Mesos-DNS Hi everybody, we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of providing Mesos-DNS with a list of masters, you point it to the Zookeeper instances. Meson-DNS will watch Zookeeper to detect the current leading master. So, while the list of Zookeeper instances is configured in a static manner, Mesos masters can be added or removed freely without restarting Mesos-DNS. The integration with Zookeeper forced to switch from -v and -vv as the flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2 (very verbose). To reduce complications because of dependencies to other packages, we have also started using godep. Please take a look at the branch https://github.com/mesosphere/mesos-dns/tree/zk and provide us with any feedback on the code or the documentation. Thanks -- Christos
Re: Zookeeper integration for Mesos-DNS
Hi, I have a question. If mesos-dns have a public ip for other application which not run in mesos? For example, we start a kafka in mesos. For other applications which not running in mesos, we use the ip as kafka config in application. So if mesos-dns have a publich ip, we could use the custom host name as kafka config after we set the dns of application to mesos dns. On Sat, Mar 21, 2015 at 8:18 AM, Christos Kozyrakis kozyr...@gmail.com wrote: Hi everybody, we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of providing Mesos-DNS with a list of masters, you point it to the Zookeeper instances. Meson-DNS will watch Zookeeper to detect the current leading master. So, while the list of Zookeeper instances is configured in a static manner, Mesos masters can be added or removed freely without restarting Mesos-DNS. The integration with Zookeeper forced to switch from -v and -vv as the flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2 (very verbose). To reduce complications because of dependencies to other packages, we have also started using godep. Please take a look at the branch https://github.com/mesosphere/mesos-dns/tree/zk and provide us with any feedback on the code or the documentation. Thanks -- Christos -- Best Regards, Haosdent Huang
Zookeeper integration for Mesos-DNS
Hi everybody, we have updated Mesos-DNS to integrate directly with Zookeeper. Instead of providing Mesos-DNS with a list of masters, you point it to the Zookeeper instances. Meson-DNS will watch Zookeeper to detect the current leading master. So, while the list of Zookeeper instances is configured in a static manner, Mesos masters can be added or removed freely without restarting Mesos-DNS. The integration with Zookeeper forced to switch from -v and -vv as the flags to control verbosity to -v=0 (default), -v=1 (verbose), and -v=2 (very verbose). To reduce complications because of dependencies to other packages, we have also started using godep. Please take a look at the branch https://github.com/mesosphere/mesos-dns/tree/zk and provide us with any feedback on the code or the documentation. Thanks -- Christos
