For instance,
No kill ack received for instance
[pub_api_oecd-rest-api-on-port-20015.marathon-196f414a-f61f-11e7-856c-f6e84742f1ef],
retrying (73402 attempts so far)
I'd say after 73402 attempts, it's time to let it go :D
On 04/04/2018 10:07 AM, Adam Cecile wrote:
Hello list !
Problem is still on-going, any hint how to fix that ? Like removing
broken app from zookeeper by hand ?
Regards, Adam.
On 03/20/2018 06:04 PM, daemeon reiydelle wrote:
I ran across a situation with the same symptoms last year (with Mesos
& Marathon) when we had network problems. The mesos task did exit
normally (eventually found same in the logs), therefore the UUID had
aged out.
<======>
"Who do you think made the first stone spear? The Asperger guy.
If you get rid of the autism genetics, there would be no Silicon Valley"
Temple Grandin
*Daemeon C.M. Reiydelle
San Francisco 1.415.501.0198
London 44 020 8144 9872*/
/*/*
*/*
On Tue, Mar 20, 2018 at 1:34 AM, Adam Cecile <[email protected]
<mailto:[email protected]>> wrote:
Hi Greg,
Yes I can confirm No kill ack received for instance
[pub_api_oecd-rest-api-on-port-20015.marathon-196f414a-f61f-11e7-856c-f6e84742f1ef],
retrying (73402 attempts so far)i cannot find this UUID in Mesos
interface.
Regards, Adam.
On 03/15/2018 05:47 PM, Greg Mann wrote:
Hi Adam,
The KILL calls are being sent to Mesos by Marathon. Since the
KILL call is being forwarded to the agent, it seems that the
Mesos master is aware of the task. Could you verify that the
tasks show up as running in the Mesos UI? You say that the tasks
don't exist anymore - how did you verify this? If the tasks show
up as running in the Mesos state, but the actual task processes
are not running on the agent, then it could indicate an issue
with the Mesos agent or executor.
Cheers,
Greg
On Wed, Mar 14, 2018 at 1:59 AM, Adam Cecile
<[email protected] <mailto:[email protected]>> wrote:
Hello,
I see two old tasks being stuck in Mesos. These tasks don't
exist anymore since ages but Mesos still tries to kill them:
Mar 14 09:56:49 mario mesos-master[23570]: I0314
09:56:49.441572 23602 master.cpp:5297] Processing KILL call
for task
'pub_api_oecd-rest-api-on-port-20015.196f414a-f61f-11e7-856c-f6e84742f1ef'
of framework 346d7333-a980-43a8-93ab-343ea12d77d7-0000
(marathon) at
[email protected]:40487
<http://[email protected]:40487>
Mar 14 09:56:49 mario mesos-master[23570]: I0314
09:56:49.441658 23602 master.cpp:5371] Telling agent
2215ab84-177b-478b-ab62-4453803fde6c-S6 at
slave(1)@10.99.50.3:5051 <http://10.99.50.3:5051>
(zelda.service.domain.com <http://zelda.service.domain.com>)
to kill task
pub_api_oecd-rest-api-on-port-20015.196f414a-f61f-11e7-856c-f6e84742f1ef
of framework 346d7333-a980-43a8-93ab-343ea12d77d7-0000
(marathon) at
[email protected]:40487
<http://[email protected]:40487>
Mar 14 09:57:09 mario mesos-master[23570]: I0314
09:57:09.441529 23607 master.cpp:5297] Processing KILL call
for task
'pub_api_oecd-rest-api-on-port-20015.196f414a-f61f-11e7-856c-f6e84742f1ef'
of framework 346d7333-a980-43a8-93ab-343ea12d77d7-0000
(marathon) at
[email protected]:40487
<http://[email protected]:40487>
Mar 14 09:57:09 mario mesos-master[23570]: I0314
09:57:09.441617 23607 master.cpp:5371] Telling agent
2215ab84-177b-478b-ab62-4453803fde6c-S6 at
slave(1)@10.99.50.3:5051 <http://10.99.50.3:5051>
(zelda.service.domain.com <http://zelda.service.domain.com>)
to kill task
pub_api_oecd-rest-api-on-port-20015.196f414a-f61f-11e7-856c-f6e84742f1ef
of framework 346d7333-a980-43a8-93ab-343ea12d77d7-0000
(marathon) at
[email protected]:40487
<http://[email protected]:40487>
Could you please tell me how to "purge" them from Mesos master ?
Thanks in advance,
Adam.
