[jira] [Commented] (MESOS-2602) Provide a way to "push" cluster state updates to a registered service.
[ https://issues.apache.org/jira/browse/MESOS-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15390471#comment-15390471 ] Deshi Xiao commented on MESOS-2602: --- Cool > Provide a way to "push" cluster state updates to a registered service. > -- > > Key: MESOS-2602 > URL: https://issues.apache.org/jira/browse/MESOS-2602 > Project: Mesos > Issue Type: Improvement >Reporter: Christos Kozyrakis >Assignee: Zhitao Li > Fix For: 1.0.0 > > > At the moment, service discovery systems like Mesos-DNS must periodically > pull the cluster state through state.json. This is extremely wasteful and > will not scale to large clusters. If the polling interval is low, the master > load will grow significantly. If the polling interval is high, there will be > added latency to service discovery. Moreover, the way state.json works right > now, one keeps reading the same information over and over again, including > info about about tasks no longer running. > We can design an endpoint that allows a "push" approach for state > information. Here is one of the possible ways to set it up: > - a service can hit the end point at (re)start to get information for all > currently running tasks. > - a service can also register itself to get receive updates to task state > beyond that (ie, notifications of tasks starting/ending/etc). We may want to > add some qualifiers here, since service discovery systems care only about > certain types of updates. > This can be implemented through direct messaging, through a message queue, by > putting messages in etcd/zookeeper, etc. We should pick the way that is most > scalable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2602) Provide a way to "push" cluster state updates to a registered service.
[ https://issues.apache.org/jira/browse/MESOS-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15384349#comment-15384349 ] Deshi Xiao commented on MESOS-2602: --- etcd /ZK is preferred way. what proposal on the action? do we have the design document? > Provide a way to "push" cluster state updates to a registered service. > --- > > Key: MESOS-2602 > URL: https://issues.apache.org/jira/browse/MESOS-2602 > Project: Mesos > Issue Type: Improvement >Reporter: Christos Kozyrakis > > At the moment, service discovery systems like Mesos-DNS must periodically > pull the cluster state through state.json. This is extremely wasteful and > will not scale to large clusters. If the polling interval is low, the master > load will grow significantly. If the polling interval is high, there will be > added latency to service discovery. Moreover, the way state.json works right > now, one keeps reading the same information over and over again, including > info about about tasks no longer running. > We can design an endpoint that allows a "push" approach for state > information. Here is one of the possible ways to set it up: > - a service can hit the end point at (re)start to get information for all > currently running tasks. > - a service can also register itself to get receive updates to task state > beyond that (ie, notifications of tasks starting/ending/etc). We may want to > add some qualifiers here, since service discovery systems care only about > certain types of updates. > This can be implemented through direct messaging, through a message queue, by > putting messages in etcd/zookeeper, etc. We should pick the way that is most > scalable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
