[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279313#comment-16279313 ] Vinod Kone commented on MESOS-1739: --- Design doc: https://docs.google.com/document/d/1iOENs0JoXPc7sf1NDBCR2tPJ_KxwU4lLtr53SrE5U3Q/edit > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly >Assignee: Benno Evers > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177221#comment-16177221 ] Zhitao Li commented on MESOS-1739: -- Ping on this too. I'm willing to work on this in the next couple of months and push this to happen. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[
https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290327#comment-15290327
]
Deshi Xiao commented on MESOS-1739:
---
log adam's describe to understand the issue's background
{quote}
bq. Adam B 7 hours, 21 minutes ago (五月 18, 2016, 7:18 p.m.)
Desired behavior: Operator can kill a slave process and restart it with new
--attributes. Existing tasks will continue to run. No TASK_LOST or SLAVE_LOST
message is sent. The slaveId remains the same. Outstanding offers from that
slave will be rescinded, and those offers will be remade with the updated
attributes.
Current behavior 1: Operator shuts down a slave process, and restarts with
--recover=cleanup, which kills all its tasks, clears the work_dir, and notifies
the master that the old slaveId is "shutdown" and will never be reused again
(SLAVE_LOST, offers rescinded, TASK_KILLED/LOST). Operator then restarts the
slave with new --attributes, it gets a new slaveId, and new offers will be made
with the new slaveId and updated attributes.
Current behavior 2: Slave process dies/killed and tries to restart with new
--attributes. Errors on recovery.
Current behavior 3: Slave process dies/killed and doesn't reregister in
slave_ping_timeout*max_slave_ping_timeouts (90s). Master considers it gone,
sends SLAVE_LOST, TASK_LOST. Future attempts to reregister with the same
slaveId fail. Slave must be cleaned up (tasks killed, work_dir removed) so it
can register with a new slaveId (and new attributes).
{quote}
> Allow slave reconfiguration on restart
> --
>
> Key: MESOS-1739
> URL: https://issues.apache.org/jira/browse/MESOS-1739
> Project: Mesos
> Issue Type: Epic
>Reporter: Patrick Reilly
> Labels: external-volumes, mesosphere, myriad
>
> Make it so that either via a slave restart or a out of process "reconfigure"
> ping, the attributes and resources of a slave can be updated to be a superset
> of what they used to be.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259825#comment-15259825 ] Klaus Ma commented on MESOS-1739: - Agree to send notification to the the frameworks. For the executors/tasks, will we kill them if slave's attributes changed? I think it's better to keep them running: 1. update slave's attributes, 2. notify frameworks & reconcile tasks, 3. slave killed tasks if necessary; no impact to other executor/tasks. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15258458#comment-15258458 ] Vinod Kone commented on MESOS-1739: --- Not just tasks, but reservations/volumes/outstanding-offers. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257669#comment-15257669 ] Adam B commented on MESOS-1739: --- > send the event to all the frameworks which have tasks on the agent? Yes, that's what we'd have to do. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257662#comment-15257662 ] Qian Zhang commented on MESOS-1739: --- But how can we know which frameworks we should send the event to? I think we do not know which tasks are impacted by the agent attribute change since there is no dependent attribute in TaskInfo. Or we just simply send the event to all the frameworks which have tasks on the agent? > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15235678#comment-15235678 ] Adam B commented on MESOS-1739: --- This is why we suggest that changes like this will need to notify (all?) frameworks of the change in attributes, so the framework can make the right choice about what to do with its tasks based on the new information. I'm not sure, however, how we should handle frameworks that don't understand the new "attributes changed" message. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[
https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15234997#comment-15234997
]
Deshi Xiao commented on MESOS-1739:
---
log yujie's comment here:
{quote}
This is a high level question: I am now sure if adding attributes is safe or
not. For instance, my framework has the following rule: only schedule tasks to
agents that do not have attribute "not_safe". Now, say agent A is initially
without that attribute. My framework lands several tasks on that agent. Later,
when agent restarts, the operator adds the new attribute "not_safe". Suddently,
i have tasks running on unsafe boxes. oops.
{quote}
> Allow slave reconfiguration on restart
> --
>
> Key: MESOS-1739
> URL: https://issues.apache.org/jira/browse/MESOS-1739
> Project: Mesos
> Issue Type: Epic
>Reporter: Patrick Reilly
> Labels: external-volumes, mesosphere, myriad
>
> Make it so that either via a slave restart or a out of process "reconfigure"
> ping, the attributes and resources of a slave can be updated to be a superset
> of what they used to be.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[
https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15217617#comment-15217617
]
Deshi Xiao commented on MESOS-1739:
---
in [Design Doc: Updating
SlaveInfo|https://docs.google.com/document/d/1PWv7YIdV3nN2l1oUW7Nybm4KdnxZ7Px2JGc5UM-PKoQ/edit#]
it statement
{quote}
Currently this can only be handled by stopping / draining a mesos slave
entirely (Killing all of its running jobs), removing it from the cluster, then
bringing it back up as a brand new slave.
{quote}
but in my testing it is not true. https://reviews.apache.org/r/45474/
> Allow slave reconfiguration on restart
> --
>
> Key: MESOS-1739
> URL: https://issues.apache.org/jira/browse/MESOS-1739
> Project: Mesos
> Issue Type: Epic
>Reporter: Patrick Reilly
> Labels: external-volumes, mesosphere, myriad
>
> Make it so that either via a slave restart or a out of process "reconfigure"
> ping, the attributes and resources of a slave can be updated to be a superset
> of what they used to be.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15217508#comment-15217508 ] Deshi Xiao commented on MESOS-1739: --- this issue is non-trivial change. so i carefully add a test case to understand the issue's scope. anyone can shepherd me to correct way? https://reviews.apache.org/r/45474/ > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15217341#comment-15217341 ] Deshi Xiao commented on MESOS-1739: --- in [Design Doc: Updating SlaveInfo|https://docs.google.com/document/d/1PWv7YIdV3nN2l1oUW7Nybm4KdnxZ7Px2JGc5UM-PKoQ/edit#heading=h.joukcb7ohf8y] 's Coding Phases and Options section, do i can update the https://reviews.apache.org/r/25525 as phase 1 result? the PR is outdate. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216340#comment-15216340 ] Yan Xu commented on MESOS-1739: --- Yes. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15215909#comment-15215909 ] Deshi Xiao commented on MESOS-1739: --- hi [~adam-mesos], [~xujyan], [~greggomann] I wonder to konw Offers if contain attributes? > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15215860#comment-15215860 ] Deshi Xiao commented on MESOS-1739: --- Hi Greg, I have no specific use case on changing way. thanks for your remind. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214911#comment-15214911 ] Adam B commented on MESOS-1739: --- Probably also want to rescind all outstanding offers from that agent, so that new offers can be generated with the updated attributes. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214894#comment-15214894 ] Yan Xu commented on MESOS-1739: --- Agreed. Therefore for a more graceful mechanism to notify the framework, I think we can send an event to the framework for each task that the agent attribute change impacts about the has changed condition that it runs under, this can be done via a special status update. The framework can then choose to kill the task. What do you think? > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214659#comment-15214659 ] Adam B commented on MESOS-1739: --- But even changing attributes to a superset (or changing existing values) could require killing tasks, depending on the framework. Imagine your framework is running sensitive tasks on a node, and then the operator tags the node with the "public_internet_access=true" attribute, because the node is now in the open. You would want to be alerted so you could kill/move your sensitive tasks, even though it's a new attribute. For resources, adding new resources would not require frameworks to be notified (beyond the existing offer mechanism) nor require tasks to be killed, because existing tasks are not consuming those resources. Removing resources could require killing tasks, if there are not enough resources left after the change to keep running all tasks. Or the agent might just prevent the operator from reducing resources below current consumption. Adding/changing/removing attributes, however, requires frameworks to be notified. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214389#comment-15214389 ] Greg Mann commented on MESOS-1739: -- For a first step, one approach would be to limit the scope of this work that it's only possible to change the slave resources/attributes to a superset of their previous values. As you see in review #25525, that was the original approach because it makes things much simpler. In fact, though it may not be *strictly* necessary to kill tasks if attributes are removed from a slave, consider the following scenario: a task was started on a slave with a particular attribute because that attribute indicates that the slave has access to a certain region of the network. If the attribute is removed because this region is no longer accessible, then it would make sense to kill the task which has that dependency. In this case, the master could notify the framework that the attributes were changed and the framework could take action as appropriate. So I think it might make sense to initially just implement a reconfiguration to a superset of previous attributes/resources. Do you have a specific use case for changing attributes/resources to a *subset* of their previous values? > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213730#comment-15213730 ] Deshi Xiao commented on MESOS-1739: --- in first try patch: https://reviews.apache.org/r/25525/diff/13#1 i only found a superset logic, if we want to change the attributes, how can i do it? so i think resources cannot be changed without kill old tasks. only attributes is satisfied with mutable purpose. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213720#comment-15213720 ] Deshi Xiao commented on MESOS-1739: --- Based on Vinod Kone's Comments on MESOS-1739, i suggest only update Attributes only. the Resources is very host-relative parameters, use currently restart policy is enough to support our purpose. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212173#comment-15212173 ] Vinod Kone commented on MESOS-1739: --- Glad to hear you are interested in this. Unfortunately I do not have cycles to shepherd this. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15211658#comment-15211658 ] Deshi Xiao commented on MESOS-1739: --- [~vi...@twitter.com] i would like to working on it. firstly we need get the design docs done. i notice the http api propose is accepted, does it mean we support api trigger to change the configuration? i have read your ideas on above comments, you like to propose: 1. Lets do the reconfiguration of a slave via a restart (at least for now) instead of via API endpoint. in our requirement, we prefer API to update the attributes. another properties such as roles, i agree with your idea. so i suggest i can identify which config can use restart agent way, and else another config can use API to update. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15024853#comment-15024853 ] Greg Mann commented on MESOS-1739: -- I'm removing myself from this ticket for now, since I've gotten caught up in other work. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly >Assignee: Greg Mann > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972146#comment-14972146 ] Vinod Kone commented on MESOS-1739: --- Great to hear. Yea. Will be happy to shepherd. > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly >Assignee: Greg Mann > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972043#comment-14972043 ] Greg Mann commented on MESOS-1739: -- I'd like to have a go at getting this thing pushed through. [~vinodkone], are you still interested in shepherding? I've read through the existing patch and reviews; I can try to come up with a solution to the repeated re-registration problem outlined above. Once I have an idea in mind, would you like me to explain my plan in a small design doc or just here via comments? > Allow slave reconfiguration on restart > -- > > Key: MESOS-1739 > URL: https://issues.apache.org/jira/browse/MESOS-1739 > Project: Mesos > Issue Type: Epic >Reporter: Patrick Reilly >Assignee: Greg Mann > Labels: external-volumes, mesosphere, myriad > > Make it so that either via a slave restart or a out of process "reconfigure" > ping, the attributes and resources of a slave can be updated to be a superset > of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536280#comment-14536280 ] Cody Maloney commented on MESOS-1739: - The biggest thing which came up in my old patchset was race conditions around re-registering in how the mesos registerSlave / reregisterSlave code is setup which probably will need some structural reworking. The case that was broken in my patch set is when a slave tries to register multiple times because it hasn't gotten a response from the master yet, and 1+ of those retries aren't identical to the first because they contain different resources / attributes (The slave started re-registration, then was restarted with new attributes before the master fully processed it), the master doesn't notice and just discards them as repeats. Allow slave reconfiguration on restart -- Key: MESOS-1739 URL: https://issues.apache.org/jira/browse/MESOS-1739 Project: Mesos Issue Type: Epic Reporter: Patrick Reilly Assignee: Cody Maloney Labels: mesosphere Make it so that either via a slave restart or a out of process reconfigure ping, the attributes and resources of a slave can be updated to be a superset of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536242#comment-14536242 ] Joe Smith commented on MESOS-1739: -- Howdy all, What's the status of this? This change would greatly increase flexibility for us operators! Thanks, Joe Allow slave reconfiguration on restart -- Key: MESOS-1739 URL: https://issues.apache.org/jira/browse/MESOS-1739 Project: Mesos Issue Type: Epic Reporter: Patrick Reilly Assignee: Cody Maloney Make it so that either via a slave restart or a out of process reconfigure ping, the attributes and resources of a slave can be updated to be a superset of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14536250#comment-14536250 ] Adam B commented on MESOS-1739: --- [~cmaloney] created a design doc and a prototype, but hasn't had time to revisit it yet. Maybe somebody else should pick it up. We agree that this is very important, along with it's FrameworkInfo corollary MESOS-703 which just got some recent attention. We could try to get a phase 1 implemented in Mesos 0.23 if somebody has the time. Allow slave reconfiguration on restart -- Key: MESOS-1739 URL: https://issues.apache.org/jira/browse/MESOS-1739 Project: Mesos Issue Type: Epic Reporter: Patrick Reilly Assignee: Cody Maloney Labels: mesosphere Make it so that either via a slave restart or a out of process reconfigure ping, the attributes and resources of a slave can be updated to be a superset of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152431#comment-14152431 ] Cody Maloney commented on MESOS-1739: - [Design document|https://docs.google.com/document/d/1PWv7YIdV3nN2l1oUW7Nybm4KdnxZ7Px2JGc5UM-PKoQ/edit?usp=sharing] Allow slave reconfiguration on restart -- Key: MESOS-1739 URL: https://issues.apache.org/jira/browse/MESOS-1739 Project: Mesos Issue Type: Improvement Reporter: Patrick Reilly Assignee: Cody Maloney Make it so that either via a slave restart or a out of process reconfigure ping, the attributes and resources of a slave can be updated to be a superset of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129485#comment-14129485 ] Patrick Reilly commented on MESOS-1739: --- [~vinodkone] I've gone ahead and closed https://reviews.apache.org/r/25111/ I'll have [~cmaloney] submit a new review board shortly. Allow slave reconfiguration on restart -- Key: MESOS-1739 URL: https://issues.apache.org/jira/browse/MESOS-1739 Project: Mesos Issue Type: Improvement Reporter: Patrick Reilly Assignee: Cody Maloney Make it so that either via a slave restart or a out of process reconfigure ping, the attributes and resources of a slave can be updated to be a superset of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1739) Allow slave reconfiguration on restart
[ https://issues.apache.org/jira/browse/MESOS-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129546#comment-14129546 ] Cody Maloney commented on MESOS-1739: - [~vinodkone]: New review request (https://reviews.apache.org/r/25525/). Updated the bug title. Tests now pass. All functionality is there. All comments are incorporated except the patch still allows both resources and attributes to be set to supersets of what they currently are. In the case where someone has a setup where they have a critical negative attribute check, they should be aware of that and just not ever add that attribute at runtime (They can always fully kill the slave then restart). Changing the recover behavior in this case doesn't break their setups, and there are a number of cases where we would like increasing attribute sets. The check is in one place, and would be one line to change / remove / make be identical if that is a hard requirement. Allow slave reconfiguration on restart -- Key: MESOS-1739 URL: https://issues.apache.org/jira/browse/MESOS-1739 Project: Mesos Issue Type: Improvement Reporter: Patrick Reilly Assignee: Cody Maloney Make it so that either via a slave restart or a out of process reconfigure ping, the attributes and resources of a slave can be updated to be a superset of what they used to be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
