[
https://issues.apache.org/jira/browse/MESOS-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325264#comment-15325264
]
Anand Mazumdar commented on MESOS-5536:
---------------------------------------
Thanks for reporting this issue [~janisz]. From the logs, This looks related to
MESOS-5380 that we recently fixed. Can you try this with Mesos 0.28.2?
> Completed executors presented as alive
> --------------------------------------
>
> Key: MESOS-5536
> URL: https://issues.apache.org/jira/browse/MESOS-5536
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 0.28.0
> Environment: Ubuntu 14.04.3 LTS
> Reporter: Tomasz Janiszewski
>
> I'm running Mesos 0.28.0. Mesos {{slave(1)/state}} endpoint returns some
> completed executors not in frameworks.completed_executors but in
> frameworks.executors. Alsa this executor presents in {{monitor/statistics}}
> {code:JavaScript:title=slave(1)/state}
> {
> "attributes": {...},
> "completed_frameworks": [],
> "flags": {...},
> "frameworks": [
> {
> "checkpoint": true,
> "completed_executors": [...],
> "executors": [
> {
> "queued_tasks": [],
> "tasks": [],
> "completed_tasks": [
> {
> "discovery": {...},
> "executor_id": "",
> "framework_id":
> "f65b163c-0faf-441f-ac14-91739fa4394c-0000",
> "id":
> "service.a3b609b8-27ec-11e6-8044-02c89eb9127e",
> "labels": [...],
> "name": "service",
> "resources": {...},
> "slave_id":
> "ef232fd9-5114-4d8f-adc3-1669c1e6fdc5-S13",
> "state": "TASK_KILLED",
> "statuses": []
> }
> ],
> "container": "ead42e63-ac92-4ad0-a99c-4af9c3fa5e31",
> "directory": "...",
> "id": "service.a3b609b8-27ec-11e6-8044-02c89eb9127e",
> "name": "Command Executor (Task:
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e) (Command: sh -c 'cd
> service...')",
> "resources": {...},
> "source": "service.a3b609b8-27ec-11e6-8044-02c89eb9127e"
>
> },
> ...
> ],
> }
> ],
> "git_sha": "961edbd82e691a619a4c171a7aadc9c32957fa73",
> "git_tag": "0.28.0",
> "version": "0.28.0",
> ...
> }
> {code}
> {code:title="var/log/mesos/mesos-slave.INFO"}
> 13:33:19.479182 [slave.cpp:1361] Got assigned task
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e for framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000
> 13:33:19.482566 [slave.cpp:1480] Launching task
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e for framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000
> 13:33:19.483921 [paths.cpp:528] Trying to chown
> '/tmp/mesos/slaves/ef232fd9-5114-4d8f-adc3-1669c1e6fdc5-S13/frameworks/f65b163c-0faf-441f-ac14-91739fa4394c-0000/executors/service.a3b609b8-27ec-11e6-8044-02c89eb9127e/runs/ead42e63-ac92-4ad0-a99c-4af9c3fa5e31'
> to user 'mesosuser'
> 13:33:19.504173 [slave.cpp:5367] Launching executor
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000 with resources cpus(*):0.1;
> mem(*):32 in work directory
> '/tmp/mesos/slaves/ef232fd9-5114-4d8f-adc3-1669c1e6fdc5-S13/frameworks/f65b163c-0faf-441f-ac14-91739fa4394c-0000/executors/service.a3b609b8-27ec-11e6-8044-02c89eb9127e/runs/ead42e63-ac92-4ad0-a99c-4af9c3fa5e31'
> 13:33:19.505537 [containerizer.cpp:666] Starting container
> 'ead42e63-ac92-4ad0-a99c-4af9c3fa5e31' for executor
> 'service.a3b609b8-27ec-11e6-8044-02c89eb9127e' of framework
> 'f65b163c-0faf-441f-ac14-91739fa4394c-0000'
> 13:33:19.505734 [slave.cpp:1698] Queuing task
> 'service.a3b609b8-27ec-11e6-8044-02c89eb9127e' for executor
> 'service.a3b609b8-27ec-11e6-8044-02c89eb9127e' of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000
> ...
> 13:33:19.977483 [containerizer.cpp:1118] Checkpointing executor's forked pid
> 25576 to
> '/tmp/mesos/meta/slaves/ef232fd9-5114-4d8f-adc3-1669c1e6fdc5-S13/frameworks/f65b163c-0faf-441f-ac14-91739fa4394c-0000/executors/service.a3b609b8-27ec-11e6-8044-02c89eb9127e/runs/ead42e63-ac92-4ad0-a99c-4af9c3fa5e31/pids/forked.pid'
> 13:33:35.775195 [slave.cpp:1891] Asked to kill task
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000
> 13:33:35.775645 [slave.cpp:3002] Handling status update TASK_KILLED (UUID:
> eba64915-7df2-483d-8982-a9a46a48a81b) for task
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000 f
> rom @0.0.0.0:0
> 13:33:35.778105 [cpushare.cpp:389] Updated 'cpu.shares' to 102 (cpus 0.1)
> for container ead42e63-ac92-4ad0-a99c-4af9c3fa5e31
> 13:33:35.778488 [disk.cpp:169] Updating the disk resources for container
> ead42e63-ac92-4ad0-a99c-4af9c3fa5e31 to cpus(*):0.1; mem(*):32
> 13:33:35.780349 [mem.cpp:353] Updated 'memory.soft_limit_in_bytes' to 32MB
> for container ead42e63-ac92-4ad0-a99c-4af9c3fa5e3
> 1
> 13:33:35.782573 [status_update_manager.cpp:320] Received status update
> TASK_KILLED (UUID: eba64915-7df2-483d-8982-a9a46a48a8
> 1b) for task service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
> f65b163c-0faf-441f-ac14-9173
> 9fa4394c-0000
> 13:33:35.783860 [status_update_manager.cpp:824] Checkpointing UPDATE for
> status update TASK_KILLED (UUID: eba64915-7df2-483d-8982-a9a46a48a81b) for
> task service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000
> 13:33:35.788767 [slave.cpp:3400] Forwarding the update TASK_KILLED (UUID:
> eba64915-7df2-483d-8982-a9a46a48a81b) for task
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000 to [email protected]:5050
> 13:33:35.917932 [status_update_manager.cpp:392] Received status update
> acknowledgement (UUID: eba64915-7df2-483d-8982-a9a46a48a81b) for task
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000
> 13:33:35.918143 [status_update_manager.cpp:824] Checkpointing ACK for status
> update TASK_KILLED (UUID: eba64915-7df2-483d-8982-a9a46a48a81b) for task
> service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000
> ...
> 13:33:39.031054 [slave.cpp:2643] Got registration for executor
> 'service.a3b609b8-27ec-11e6-8044-02c89eb9127e' of framework
> f65b163c-0faf-441f-ac14-91739fa4394c-0000 from executor(1)@127.0.0.1:60083
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
