[jira] [Comment Edited] (MESOS-10084) Detecting whether executor is generated for command task should work when the launcher_dir changes

2020-01-27 Thread Benjamin Bannier (Jira)


[ 
https://issues.apache.org/jira/browse/MESOS-10084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022937#comment-17022937
 ] 

Benjamin Bannier edited comment on MESOS-10084 at 1/27/20 2:27 PM:
---

{{1.5.x}}
{noformat}
commit 2f2146ac61abd54bc3296d8cbb5429cd10584db1
Author: Benjamin Bannier 
Date:   Thu Jan 23 14:19:57 2020 +0100

Remembered whether an executor was agent-generated.

This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.

Review: https://reviews.apache.org/r/72035/

commit 6294c319047f4c23ac28c9f20c39c88dfad2a66b
Author: Benjamin Bannier 
Date:   Thu Jan 23 14:19:51 2020 +0100

Sync'd whether an executor was generated to and from disk.

This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.

Review: https://reviews.apache.org/r/72034/

commit 095021dcadde9a19ed3298c65dcbed303651faa8
Author: Benjamin Bannier 
Date:   Thu Jan 23 14:19:43 2020 +0100

Decoupled detection of generated executors from Mesos install location.

We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`.  This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.

This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.

Review: https://reviews.apache.org/r/72033/
{noformat}

{{1.6.x}}:
{noformat}
commit 7e4d380c11c20f4b9e20b06f6b6c67a4657af24b
Author: Benjamin Bannier 
Date:   Thu Jan 23 14:19:57 2020 +0100

Remembered whether an executor was agent-generated.

This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.

Review: https://reviews.apache.org/r/72035/

commit 305f2b5e88ed9256b60a02afbdad06e2333937b7
Author: Benjamin Bannier 
Date:   Thu Jan 23 14:19:51 2020 +0100

Sync'd whether an executor was generated to and from disk.

This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agent-generated executors with a heuristic,
but will adapt that heuristic in a follow-up path making us of the
additional state we now persist.

Review: https://reviews.apache.org/r/72034/

commit efbb1e697af7082f3da1d509df9cd25ccbe0aab8
Author: Benjamin Bannier 
Date:   Thu Jan 23 14:19:43 2020 +0100

Decoupled detection of generated executors from Mesos install location.

We previously were detecting executors generated for command tasks by
checking whether their command matched the full path of
`mesos-executor`.  This approach can lead to misdetection if e.g.,
during an upgrade a new installation location is choosen.

This patch adjusts the heuristic by now only relying on the fact that
the executor command should end in `mesos-executor`. In order to cut
down on false positives we now additionally check that the executor name
looks similar to the ones we generate for command tasks.

Review: https://reviews.apache.org/r/72033/
{noformat}

{{1.7.x}}:
{noformat}
commit 47f2a11e6ade71c455efe285898b52c256681e31
Author: Benjamin Bannier 
Date:   Thu Jan 23 14:19:57 2020 +0100

Remembered whether an executor was agent-generated.

This patch adds code to pass on whether was generated in the agent from
the point where the executor is generated to the point where we create
an actual `slave::Executor` instance. This allows us to persist this
information in the executor state.

Review: https://reviews.apache.org/r/72035/

commit 989ccb5c04c25dfde2d9c530cc84218268c403a6
Author: Benjamin Bannier 
Date:   Thu Jan 23 14:19:51 2020 +0100

Sync'd whether an executor was generated to and from disk.

This patch introduces an `ExecutorState` variable signifying whether an
executor was generated by the agent (and is thus unknown to the master).
Currently we still detect agen

[jira] [Comment Edited] (MESOS-10084) Detecting whether executor is generated for command task should work when the launcher_dir changes

2020-01-21 Thread Benjamin Bannier (Jira)


[ 
https://issues.apache.org/jira/browse/MESOS-10084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015863#comment-17015863
 ] 

Benjamin Bannier edited comment on MESOS-10084 at 1/21/20 12:21 PM:


Reviews:
https://reviews.apache.org/r/72033/
https://reviews.apache.org/r/72034/
https://reviews.apache.org/r/72035/


was (Author: bbannier):
Reviews:
https://reviews.apache.org/r/72002/
https://reviews.apache.org/r/72003/

> Detecting whether executor is generated for command task should work when the 
> launcher_dir changes
> --
>
> Key: MESOS-10084
> URL: https://issues.apache.org/jira/browse/MESOS-10084
> Project: Mesos
>  Issue Type: Bug
>Reporter: Andrei Sekretenko
>Assignee: Benjamin Bannier
>Priority: Critical
>
> As currently implemented, on recovery Mesos agent determines that the 
> executor is generated for command task by comparing the executor command with 
> a current path to Mesos executor:
> https://github.com/apache/mesos/blob/1.7.x/src/slave/slave.cpp#L9635
> During upgrade of production cluster we observed this check to break due to 
> the new launcher_dir being different from the one of checkpointed executor.
> This can cause problems of various kind: for example, after such upgrade, 
> Mesos master can begin to treat the checkpointed command executors as subject 
> to resource quota.
> Design considerations:
>  - proper solution is to checkpoint the flag indicating whether the executor 
> is a command/docker one.
>  - for correct upgrade from older Mesos versions, we will need some kind of 
> workaround to detect command executors after upgrade; the workaround logic 
> should be skipped if there is a checkpointed flag.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)