[jira] [Comment Edited] (MESOS-6400) Not able to remove Orphan Tasks
[
https://issues.apache.org/jira/browse/MESOS-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15593665#comment-15593665
]
kasim edited comment on MESOS-6400 at 10/21/16 1:53 AM:
[~gilbert], there are two machines's logs, from 2016.9.26 to 2016.10.21
https://drive.google.com/open?id=0B1ULA2gXggVwOWQwLUxCYmdoWnc
https://drive.google.com/open?id=0B1ULA2gXggVwRmJCRmI1Qm1OOEU
1. Mesos and Marathon out of sync happen on someday in 2016.10.1 - 2016.10.7
2. I restart Marathon in 2016.10.8
was (Author: mithril):
[~gilbert] There are two machines's logs, from 2016.9.26 to 2016.10.21
https://drive.google.com/open?id=0B1ULA2gXggVwOWQwLUxCYmdoWnc
https://drive.google.com/open?id=0B1ULA2gXggVwRmJCRmI1Qm1OOEU
1. Mesos and Marathon out of sync happen on someday in 2016.10.1 - 2016.10.7
2. I restart Marathon in 2016.10.8
> Not able to remove Orphan Tasks
> ---
>
> Key: MESOS-6400
> URL: https://issues.apache.org/jira/browse/MESOS-6400
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.0.1
> Environment: centos 7 x64
>Reporter: kasim
>
> The problem maybe cause by Mesos and Marathon out of sync
> https://github.com/mesosphere/marathon/issues/616
> When I found Orphan Tasks happen, I
> 1. restart marathon
> 2. marathon do not sync Orphan Tasks, but start new tasks.
> 3. Orphan Tasks still taked the resource, I have to delete them.
> 4. I find all Orphan Tasks are under framework
> `ef169d8a-24fc-41d1-8b0d-c67718937a48-`,
> curl -XGET `http://c196:5050/master/frameworks` shows that framework is
> `unregistered_frameworks`
> {code}
> {
> "frameworks": [
> .
> ],
> "completed_frameworks": [ ],
> "unregistered_frameworks": [
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-"
> ]
> }
> {code}
> 5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
> 'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
> ， but get `No framework found with specified ID`
> So I have no idea to delete Orphan Tasks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-6400) Not able to remove Orphan Tasks
[
https://issues.apache.org/jira/browse/MESOS-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15593665#comment-15593665
]
kasim edited comment on MESOS-6400 at 10/21/16 1:53 AM:
[~gilbert] There are two machines's logs, from 2016.9.26 to 2016.10.21
https://drive.google.com/open?id=0B1ULA2gXggVwOWQwLUxCYmdoWnc
https://drive.google.com/open?id=0B1ULA2gXggVwRmJCRmI1Qm1OOEU
1. Mesos and Marathon out of sync happen on someday in 2016.10.1 - 2016.10.7
2. I restart Marathon in 2016.10.8
was (Author: mithril):
There are two machines's logs, from 2016.9.26 to 2016.10.21
https://drive.google.com/open?id=0B1ULA2gXggVwOWQwLUxCYmdoWnc
https://drive.google.com/open?id=0B1ULA2gXggVwRmJCRmI1Qm1OOEU
1. Mesos and Marathon out of sync happen on someday in 2016.10.1 - 2016.10.7
2. I restart Marathon in 2016.10.8
> Not able to remove Orphan Tasks
> ---
>
> Key: MESOS-6400
> URL: https://issues.apache.org/jira/browse/MESOS-6400
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.0.1
> Environment: centos 7 x64
>Reporter: kasim
>
> The problem maybe cause by Mesos and Marathon out of sync
> https://github.com/mesosphere/marathon/issues/616
> When I found Orphan Tasks happen, I
> 1. restart marathon
> 2. marathon do not sync Orphan Tasks, but start new tasks.
> 3. Orphan Tasks still taked the resource, I have to delete them.
> 4. I find all Orphan Tasks are under framework
> `ef169d8a-24fc-41d1-8b0d-c67718937a48-`,
> curl -XGET `http://c196:5050/master/frameworks` shows that framework is
> `unregistered_frameworks`
> {code}
> {
> "frameworks": [
> .
> ],
> "completed_frameworks": [ ],
> "unregistered_frameworks": [
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-"
> ]
> }
> {code}
> 5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
> 'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
> ， but get `No framework found with specified ID`
> So I have no idea to delete Orphan Tasks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6400) Not able to remove Orphan Tasks
[
https://issues.apache.org/jira/browse/MESOS-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15593665#comment-15593665
]
kasim commented on MESOS-6400:
--
There are two machines's logs, from 2016.9.26 to 2016.10.21
https://drive.google.com/open?id=0B1ULA2gXggVwOWQwLUxCYmdoWnc
https://drive.google.com/open?id=0B1ULA2gXggVwRmJCRmI1Qm1OOEU
1. Mesos and Marathon out of sync happen on someday in 2016.10.1 - 2016.10.7
2. I restart Marathon in 2016.10.8
> Not able to remove Orphan Tasks
> ---
>
> Key: MESOS-6400
> URL: https://issues.apache.org/jira/browse/MESOS-6400
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.0.1
> Environment: centos 7 x64
>Reporter: kasim
>
> The problem maybe cause by Mesos and Marathon out of sync
> https://github.com/mesosphere/marathon/issues/616
> When I found Orphan Tasks happen, I
> 1. restart marathon
> 2. marathon do not sync Orphan Tasks, but start new tasks.
> 3. Orphan Tasks still taked the resource, I have to delete them.
> 4. I find all Orphan Tasks are under framework
> `ef169d8a-24fc-41d1-8b0d-c67718937a48-`,
> curl -XGET `http://c196:5050/master/frameworks` shows that framework is
> `unregistered_frameworks`
> {code}
> {
> "frameworks": [
> .
> ],
> "completed_frameworks": [ ],
> "unregistered_frameworks": [
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-"
> ]
> }
> {code}
> 5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
> 'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
> ， but get `No framework found with specified ID`
> So I have no idea to delete Orphan Tasks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6400) Not able to remove Orphan Tasks
[
https://issues.apache.org/jira/browse/MESOS-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587231#comment-15587231
]
kasim commented on MESOS-6400:
--
I am using marathon 1.3.0-1.0.506.el7.
Yes, I can restarted marathon get a new framework id, and start some tasks(all
duplicated of Orphan Tasks). And due to lack of resouce, it can not start all
tasks. so I'd like to remove Orphan Tasks immediately, is there any way to do ?
> Not able to remove Orphan Tasks
> ---
>
> Key: MESOS-6400
> URL: https://issues.apache.org/jira/browse/MESOS-6400
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.0.1
> Environment: centos 7 x64
>Reporter: kasim
>
> The problem maybe cause by Mesos and Marathon out of sync
> https://github.com/mesosphere/marathon/issues/616
> When I found Orphan Tasks happen, I
> 1. restart marathon
> 2. marathon do not sync Orphan Tasks, but start new tasks.
> 3. Orphan Tasks still taked the resource, I have to delete them.
> 4. I find all Orphan Tasks are under framework
> `ef169d8a-24fc-41d1-8b0d-c67718937a48-`,
> curl -XGET `http://c196:5050/master/frameworks` shows that framework is
> `unregistered_frameworks`
> {code}
> {
> "frameworks": [
> .
> ],
> "completed_frameworks": [ ],
> "unregistered_frameworks": [
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-"
> ]
> }
> {code}
> 5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
> 'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
> ， but get `No framework found with specified ID`
> So I have no idea to delete Orphan Tasks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-6400) Not able to remove Orphan Tasks
[
https://issues.apache.org/jira/browse/MESOS-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kasim updated MESOS-6400:
-
Description:
The problem maybe cause by Mesos and Marathon out of sync
https://github.com/mesosphere/marathon/issues/616
When I found Orphan Tasks happen, I
1. restart marathon
2. marathon do not sync Orphan Tasks, but start new tasks.
3. Orphan Tasks still taked the resource, I have to delete them.
4. I find all Orphan Tasks are under framework
`ef169d8a-24fc-41d1-8b0d-c67718937a48-`,
curl -XGET `http://c196:5050/master/frameworks` shows that framework is
`unregistered_frameworks`
{code}
{
"frameworks": [
.
],
"completed_frameworks": [ ],
"unregistered_frameworks": [
"ef169d8a-24fc-41d1-8b0d-c67718937a48-",
"ef169d8a-24fc-41d1-8b0d-c67718937a48-",
"ef169d8a-24fc-41d1-8b0d-c67718937a48-"
]
}
{code}
5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
， but get `No framework found with specified ID`
So I have no idea to delete Orphan Tasks
was:
The problem maybe cause by Mesos and Marathon out of sync
https://github.com/mesosphere/marathon/issues/616
When I found Orphan Tasks happen, I
1. restart marathon
2. marathon do not sync Orphan Tasks, but start new tasks.
3. Orphan Tasks still taked the resource, I have to delete them.
4. all Orphan Tasks is under `ef169d8a-24fc-41d1-8b0d-c67718937a48-`
curl -XGET `http://c196:5050/master/frameworks` shows that framework is
`unregistered_frameworks`
{code}
{
"frameworks": [
.
],
"completed_frameworks": [ ],
"unregistered_frameworks": [
"ef169d8a-24fc-41d1-8b0d-c67718937a48-",
"ef169d8a-24fc-41d1-8b0d-c67718937a48-",
"ef169d8a-24fc-41d1-8b0d-c67718937a48-"
]
}
{code}
5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
， but get `No framework found with specified ID`
So I have no idea to delete Orphan Tasks
> Not able to remove Orphan Tasks
> ---
>
> Key: MESOS-6400
> URL: https://issues.apache.org/jira/browse/MESOS-6400
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.0.1
> Environment: centos 7 x64
>Reporter: kasim
>
> The problem maybe cause by Mesos and Marathon out of sync
> https://github.com/mesosphere/marathon/issues/616
> When I found Orphan Tasks happen, I
> 1. restart marathon
> 2. marathon do not sync Orphan Tasks, but start new tasks.
> 3. Orphan Tasks still taked the resource, I have to delete them.
> 4. I find all Orphan Tasks are under framework
> `ef169d8a-24fc-41d1-8b0d-c67718937a48-`,
> curl -XGET `http://c196:5050/master/frameworks` shows that framework is
> `unregistered_frameworks`
> {code}
> {
> "frameworks": [
> .
> ],
> "completed_frameworks": [ ],
> "unregistered_frameworks": [
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-"
> ]
> }
> {code}
> 5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
> 'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
> ， but get `No framework found with specified ID`
> So I have no idea to delete Orphan Tasks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-6400) Not able to remove Orphan Tasks
[
https://issues.apache.org/jira/browse/MESOS-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kasim updated MESOS-6400:
-
Description:
The problem maybe cause by Mesos and Marathon out of sync
https://github.com/mesosphere/marathon/issues/616
When I found Orphan Tasks happen, I
1. restart marathon
2. marathon do not sync Orphan Tasks, but start new tasks.
3. but Orphan Tasks still taked the resource, have to delete them.
4. all Orphan Tasks in under `ef169d8a-24fc-41d1-8b0d-c67718937a48-`
curl -XGET `http://c196:5050/master/frameworks` shows that framework is
`unregistered_frameworks`
{code}
{
"frameworks": [
.
],
"completed_frameworks": [ ],
"unregistered_frameworks": [
"ef169d8a-24fc-41d1-8b0d-c67718937a48-",
"ef169d8a-24fc-41d1-8b0d-c67718937a48-",
"ef169d8a-24fc-41d1-8b0d-c67718937a48-"
]
}
{code}
5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
， but get `No framework found with specified ID`
So I have no idea to delete Orphan Tasks
was:
The problem maybe cause by Mesos and Marathon out of sync
https://github.com/mesosphere/marathon/issues/616
When I found Orphan Tasks happen, I
1. restart marathon
2. marathon do not sync Orphan Tasks, but start new tasks.
3. curl -XGET `http://c196:5050/master/frameworks` :
{code}
{
"frameworks": [
.
],
"completed_frameworks": [ ],
"unregistered_frameworks": [
"ef169d8a-24fc-41d1-8b0d-c67718937a48-",
"ef169d8a-24fc-41d1-8b0d-c67718937a48-",
"ef169d8a-24fc-41d1-8b0d-c67718937a48-"
]
}
{code}
all Orphan Tasks in under `ef169d8a-24fc-41d1-8b0d-c67718937a48-`
5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
， but get `No framework found with specified ID`
So I have no idea to delete Orphan Tasks
> Not able to remove Orphan Tasks
> ---
>
> Key: MESOS-6400
> URL: https://issues.apache.org/jira/browse/MESOS-6400
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 1.0.1
> Environment: centos 7 x64
>Reporter: kasim
>
> The problem maybe cause by Mesos and Marathon out of sync
> https://github.com/mesosphere/marathon/issues/616
> When I found Orphan Tasks happen, I
> 1. restart marathon
> 2. marathon do not sync Orphan Tasks, but start new tasks.
> 3. but Orphan Tasks still taked the resource, have to delete them.
> 4. all Orphan Tasks in under `ef169d8a-24fc-41d1-8b0d-c67718937a48-`
> curl -XGET `http://c196:5050/master/frameworks` shows that framework is
> `unregistered_frameworks`
> {code}
> {
> "frameworks": [
> .
> ],
> "completed_frameworks": [ ],
> "unregistered_frameworks": [
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-",
> "ef169d8a-24fc-41d1-8b0d-c67718937a48-"
> ]
> }
> {code}
> 5.Try {code}curl -XPOST http://c196:5050/master/teardown -d
> 'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-' {code}
> ， but get `No framework found with specified ID`
> So I have no idea to delete Orphan Tasks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524701#comment-15524701
]
kasim edited comment on MESOS-6248 at 9/27/16 1:21 AM:
---
Thanks, create soft link `ln -s /lib/systemd/systemd /sbin/init` work around
this error. It is weird `/sbin/init` suddentlly disappeared.
was (Author: mithril):
Thanks, create soft link `ln -s /lib/systemd/systemd /sbin/init` work around
this erorr. It is weird `/sbin/init` suddentlly disappeared.
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>Assignee: Jan Schlicht
> Fix For: 1.1.0
>
>
> There occurs an error
> {code}
> mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> {code}
> when I start `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
> {code}
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594654510807141days
> Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070
> slave.cpp:2376] Ignoring updating pid for framework
> ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist
> Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597189882008970days
> Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595 5050
> logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting
> Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before
> InitGoogleLogging() is written to STDERR
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932 4475
> main.cpp:243] Build: 2016-08-26 23:06:27 by centos
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128 4475
> main.cpp:244] Version: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138 4475
> main.cpp:247] Git tag: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144 4475
> main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127 4475
> logging.cpp:194] INFO level logging started!
> Sep 26
[jira] [Commented] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524701#comment-15524701
]
kasim commented on MESOS-6248:
--
Thanks, create soft link `ln -s /lib/systemd/systemd /sbin/init` work around
this eror. It is weird `/sbin/init` suddentlly disappeared.
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>Assignee: Jan Schlicht
> Fix For: 1.1.0
>
>
> There occurs an error
> {code}
> mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> {code}
> when I start `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
> {code}
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594654510807141days
> Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070
> slave.cpp:2376] Ignoring updating pid for framework
> ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist
> Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597189882008970days
> Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595 5050
> logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting
> Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before
> InitGoogleLogging() is written to STDERR
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932 4475
> main.cpp:243] Build: 2016-08-26 23:06:27 by centos
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128 4475
> main.cpp:244] Version: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138 4475
> main.cpp:247] Git tag: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144 4475
> main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127 4475
> logging.cpp:194] INFO level logging started!
> Sep 26 15:51:42 zk2 mesos-slave[4475]: mesos-slave:
> ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion
[jira] [Comment Edited] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524701#comment-15524701
]
kasim edited comment on MESOS-6248 at 9/27/16 1:20 AM:
---
Thanks, create soft link `ln -s /lib/systemd/systemd /sbin/init` work around
this erorr. It is weird `/sbin/init` suddentlly disappeared.
was (Author: mithril):
Thanks, create soft link `ln -s /lib/systemd/systemd /sbin/init` work around
this eror. It is weird `/sbin/init` suddentlly disappeared.
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>Assignee: Jan Schlicht
> Fix For: 1.1.0
>
>
> There occurs an error
> {code}
> mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> {code}
> when I start `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
> {code}
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594654510807141days
> Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070
> slave.cpp:2376] Ignoring updating pid for framework
> ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist
> Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597189882008970days
> Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595 5050
> logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting
> Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before
> InitGoogleLogging() is written to STDERR
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932 4475
> main.cpp:243] Build: 2016-08-26 23:06:27 by centos
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128 4475
> main.cpp:244] Version: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138 4475
> main.cpp:247] Git tag: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144 4475
> main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127 4475
> logging.cpp:194] INFO level logging started!
> Sep 26
[jira] [Issue Comment Deleted] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kasim updated MESOS-6248:
-
Comment: was deleted
(was: {code}/sbin/init {code} not exists. What it should be ? Can I fix it
manually?)
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>Assignee: Jan Schlicht
> Fix For: 1.1.0
>
>
> There occurs an error
> {code}
> mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> {code}
> when I start `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
> {code}
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594654510807141days
> Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070
> slave.cpp:2376] Ignoring updating pid for framework
> ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist
> Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597189882008970days
> Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595 5050
> logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting
> Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before
> InitGoogleLogging() is written to STDERR
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932 4475
> main.cpp:243] Build: 2016-08-26 23:06:27 by centos
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128 4475
> main.cpp:244] Version: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138 4475
> main.cpp:247] Git tag: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144 4475
> main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127 4475
> logging.cpp:194] INFO level logging started!
> Sep 26 15:51:42 zk2 mesos-slave[4475]: mesos-slave:
> ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> Sep 26 15:51:42 zk2 mesos-slave[4475]: ***
[jira] [Commented] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524628#comment-15524628
]
kasim commented on MESOS-6248:
--
{code}/sbin/init {code} not exists. What it should be ? Can I fix it manually?
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>Assignee: Jan Schlicht
> Fix For: 1.1.0
>
>
> There occurs an error
> {code}
> mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> {code}
> when I start `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
> {code}
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594654510807141days
> Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070
> slave.cpp:2376] Ignoring updating pid for framework
> ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist
> Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597189882008970days
> Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595 5050
> logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting
> Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before
> InitGoogleLogging() is written to STDERR
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932 4475
> main.cpp:243] Build: 2016-08-26 23:06:27 by centos
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128 4475
> main.cpp:244] Version: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138 4475
> main.cpp:247] Git tag: 1.0.1
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144 4475
> main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
> Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127 4475
> logging.cpp:194] INFO level logging started!
> Sep 26 15:51:42 zk2 mesos-slave[4475]: mesos-slave:
> ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> Sep 26 15:51:42 zk2 mesos-slave[4475]: ***
[jira] [Comment Edited] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522536#comment-15522536
]
kasim edited comment on MESOS-6248 at 9/26/16 9:21 AM:
---
Same error, I found
*service mesos-slave start* excute
*logged /usr/sbin/mesos-master "${args[@]:-}"*
in *function slave* in */usr/bin/mesos-init-wrapper*.
I print args out and excute directly:
{code}
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
{code}
{code}
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
{code}
was (Author: mithril):
Same error, I found
*service mesos-slave start* excute
*logged /usr/sbin/mesos-master "${args[@]:-}"* in *function slave* in
*/usr/bin/mesos-init-wrapper*.
I print args out and excute directly:
{code}
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
{code}
{code}
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
{code}
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>
> There occurs an error
> {code}
> mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> {code}
> when I start `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
> {code}
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926
[jira] [Comment Edited] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522536#comment-15522536
]
kasim edited comment on MESOS-6248 at 9/26/16 9:21 AM:
---
Same error, I found
*service mesos-slave start* excute
*logged /usr/sbin/mesos-master "${args[@]:-}"* in *function slave* in
*/usr/bin/mesos-init-wrapper*.
I print args out and excute directly:
{code}
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
{code}
{code}
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
{code}
was (Author: mithril):
Same error, I found
{code}service mesos-slave start{code} excute
{code}logged /usr/sbin/mesos-master "${args[@]:-}"{code} in {code}function
slave{code} in {code}/usr/bin/mesos-init-wrapper{code}.
I print args out and excute directly:
{code}
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
{code}
{code}
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
{code}
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>
> There occurs an error
> {code}
> mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> {code}
> when I start `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
> {code}
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2
[jira] [Comment Edited] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522536#comment-15522536
]
kasim edited comment on MESOS-6248 at 9/26/16 9:19 AM:
---
Same error, I found
{code}service mesos-slave start{code} excute
{code}logged /usr/sbin/mesos-master "${args[@]:-}"{code} in {code}function
slave{code} in {code}/usr/bin/mesos-init-wrapper{code}.
I print args out and excute directly:
{code}
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
{code}
{code}
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
{code}
was (Author: mithril):
Same error, I found
`service mesos-slave start` excute
`logged /usr/sbin/mesos-master "${args[@]:-}"` in `function slave` in
`/usr/bin/mesos-init-wrapper`.
I print args out and excute directly:
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
```
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
```
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>
> There occurs an error
> {code}
> mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.
> {code}
> when I start `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
> {code}
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2
[jira] [Updated] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kasim updated MESOS-6248:
-
Description:
There occurs an error
{code}
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
{code}
when I start `mesos-slave`.
What I do before this error happen:
- service docker stop and add a `registry-mirror`
- service docker start
- service mesos-slave start
Log as below:
{code}
Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
4.594670214119757days
Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
4.594670214119757days
Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070
slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
4.594654510807141days
Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070
slave.cpp:2376] Ignoring updating pid for framework
ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist
Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070
slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
4.594667358972014days
Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070
slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
4.594667358972014days
Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070
slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
4.594667358972014days
Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070
slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
4.594667358972014days
Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070
slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
4.594667358972014days
Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070
slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
4.597191309582847days
Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070
slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
4.597191309582847days
Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070
slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
4.597191309582847days
Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070
slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
4.597191309582847days
Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070
slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
4.597191309582847days
Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922 5070
slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
4.597191309582847days
Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734 5070
slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
4.597191309582847days
Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595 5070
slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
4.597189882008970days
Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595 5050
logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting
Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before
InitGoogleLogging() is written to STDERR
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932 4475
main.cpp:243] Build: 2016-08-26 23:06:27 by centos
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128 4475
main.cpp:244] Version: 1.0.1
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138 4475
main.cpp:247] Git tag: 1.0.1
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144 4475
main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127 4475
logging.cpp:194] INFO level logging started!
Sep 26 15:51:42 zk2 mesos-slave[4475]: mesos-slave:
../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
Sep 26 15:51:42 zk2 mesos-slave[4475]: *** Aborted at 1474876302 (unix time)
try "date -d @1474876302" if you are using GNU date ***
Sep 26 15:51:42 zk2 mesos-slave[4475]: PC: @ 0x7f795491e5f7 __GI_raise
Sep 26 15:51:42 zk2 mesos-slave[4475]: *** SIGABRT (@0x117b) received by PID
4475 (TID 0x7f79577b98c0) from PID 4475; stack trace: ***
Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f79551d8100 (unknown)
Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f795491e5f7 __GI_raise
Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f795491fce8 __GI_abort
Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f7954917566
__assert_fail_base
Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f7954917612
[jira] [Updated] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[ https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kasim updated MESOS-6248: - Description: There occurs an error `mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string& Result::error() const [with T = std::basic_string; std::string = std::basic_string]: Assertion `isError()' failed.` when I start `mesos-slave`. What I do before this error happen: - service docker stop and add a `registry-mirror` - service docker start - service mesos-slave start Log as below: Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594670214119757days Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594670214119757days Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594654510807141days Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070 slave.cpp:2376] Ignoring updating pid for framework ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597189882008970days Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595 5050 logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before InitGoogleLogging() is written to STDERR Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932 4475 main.cpp:243] Build: 2016-08-26 23:06:27 by centos Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128 4475 main.cpp:244] Version: 1.0.1 Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138 4475 main.cpp:247] Git tag: 1.0.1 Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144 4475 main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3 Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127 4475 logging.cpp:194] INFO level logging started! Sep 26 15:51:42 zk2 mesos-slave[4475]: mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string& Result::error() const [with T = std::basic_string; std::string = std::basic_string]: Assertion `isError()' failed. Sep 26 15:51:42 zk2 mesos-slave[4475]: *** Aborted at 1474876302 (unix time) try "date -d @1474876302" if you are using GNU date *** Sep 26 15:51:42 zk2 mesos-slave[4475]: PC: @ 0x7f795491e5f7 __GI_raise Sep 26 15:51:42 zk2 mesos-slave[4475]: *** SIGABRT (@0x117b) received by PID 4475 (TID 0x7f79577b98c0) from PID 4475; stack trace: *** Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f79551d8100 (unknown) Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f795491e5f7 __GI_raise Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f795491fce8 __GI_abort Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f7954917566 __assert_fail_base Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f7954917612 __GI___assert_fail Sep 26
[jira] [Comment Edited] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522536#comment-15522536
]
kasim edited comment on MESOS-6248 at 9/26/16 9:14 AM:
---
Same error, I found
`service mesos-slave start` excute
`logged /usr/sbin/mesos-master "${args[@]:-}"` in `function slave` in
`/usr/bin/mesos-init-wrapper`.
I copy the print args out and excute directly:
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
```
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
```
was (Author: mithril):
Same error, I found
service mesos-slave start excute
`logged /usr/sbin/mesos-master "${args[@]:-}"` in function slave
in `/usr/bin/mesos-init-wrapper`.
I copy the print args out and excute directly:
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
```
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
```
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>
> There is an error `mesos-slave:
> ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.` when I start
> `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
>
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
> slave.cpp:4591] Current disk usage
[jira] [Comment Edited] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522536#comment-15522536
]
kasim edited comment on MESOS-6248 at 9/26/16 9:14 AM:
---
Same error, I found
`service mesos-slave start` excute
`logged /usr/sbin/mesos-master "${args[@]:-}"` in `function slave` in
`/usr/bin/mesos-init-wrapper`.
I print args out and excute directly:
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
```
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
```
was (Author: mithril):
Same error, I found
`service mesos-slave start` excute
`logged /usr/sbin/mesos-master "${args[@]:-}"` in `function slave` in
`/usr/bin/mesos-init-wrapper`.
I copy the print args out and excute directly:
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
```
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
```
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>
> There is an error `mesos-slave:
> ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.` when I start
> `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
>
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
> slave.cpp:4591] Current disk usage
[jira] [Commented] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
[
https://issues.apache.org/jira/browse/MESOS-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522536#comment-15522536
]
kasim commented on MESOS-6248:
--
Same error, I found
service mesos-slave start excute
`logged /usr/sbin/mesos-master "${args[@]:-}"` in function slave
in `/usr/bin/mesos-init-wrapper`.
I copy the print args out and excute directly:
[root@zk2 mesos]# mesos-slave
--master=zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
--log_dir=/var/log/mesos --containerizers=docker,mesos
--executor_registration_timeout=10mins --hostname=10.142.55.196
--ip=10.142.55.196 --work_dir=/var/lib/mesos --no-systemd_enable_support
```
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0926 18:12:24.743870 3658 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
I0926 18:12:24.744038 3658 main.cpp:244] Version: 1.0.1
I0926 18:12:24.744047 3658 main.cpp:247] Git tag: 1.0.1
I0926 18:12:24.744055 3658 main.cpp:251] Git SHA:
3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
I0926 18:12:24.745198 3658 logging.cpp:194] INFO level logging started!
mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string&
Result::error() const [with T = std::basic_string; std::string =
std::basic_string]: Assertion `isError()' failed.
*** Aborted at 1474884744 (unix time) try "date -d @1474884744" if you are
using GNU date ***
PC: @ 0x7f4cc2fc25f7 __GI_raise
*** SIGABRT (@0xe4a) received by PID 3658 (TID 0x7f4cc5e5d8c0) from PID 3658;
stack trace: ***
@ 0x7f4cc387c100 (unknown)
@ 0x7f4cc2fc25f7 __GI_raise
@ 0x7f4cc2fc3ce8 __GI_abort
@ 0x7f4cc2fbb566 __assert_fail_base
@ 0x7f4cc2fbb612 __GI___assert_fail
@ 0x7f4cc45dfe4a Result<>::error()
@ 0x7f4cc4cde9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
@ 0x7f4cc4cdf0bd systemd::exists()
@ 0x41063c main
@ 0x7f4cc2faeb15 __libc_start_main
@ 0x412099 (unknown)
Aborted
```
> mesos-slave cannot start , Assertion `isError()' failed
> ---
>
> Key: MESOS-6248
> URL: https://issues.apache.org/jira/browse/MESOS-6248
> Project: Mesos
> Issue Type: Bug
> Environment: mesos 1.01 centos 7
>Reporter: kasim
>
> There is an error `mesos-slave:
> ../../3rdparty/stout/include/stout/result.hpp:134: const string&
> Result::error() const [with T = std::basic_string; std::string =
> std::basic_string]: Assertion `isError()' failed.` when I start
> `mesos-slave`.
> What I do before this error happen:
> - service docker stop and add a `registry-mirror`
> - service docker start
> - service mesos-slave start
> Log as below:
>
> Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594670214119757days
> Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594654510807141days
> Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070
> slave.cpp:2376] Ignoring updating pid for framework
> ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist
> Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070
> slave.cpp:4591] Current disk usage 24.36%. Max allowed age:
> 4.594667358972014days
> Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070
> slave.cpp:4591] Current disk usage 24.33%. Max allowed age:
> 4.597191309582847days
> Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070
> slave.cpp:4591] Current
[jira] [Created] (MESOS-6248) mesos-slave cannot start , Assertion `isError()' failed
kasim created MESOS-6248: Summary: mesos-slave cannot start , Assertion `isError()' failed Key: MESOS-6248 URL: https://issues.apache.org/jira/browse/MESOS-6248 Project: Mesos Issue Type: Bug Environment: mesos 1.01 centos 7 Reporter: kasim There is an error `mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string& Result::error() const [with T = std::basic_string; std::string = std::basic_string]: Assertion `isError()' failed.` when I start `mesos-slave`. What I do before this error happen: - service docker stop and add a `registry-mirror` - service docker start - service mesos-slave start Log as below: Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594670214119757days Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594670214119757days Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594654510807141days Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425 5070 slave.cpp:2376] Ignoring updating pid for framework ef169d8a-24fc-41d1-8b0d-c67718937a48- because it does not exist Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061 5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595 5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597189882008970days Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595 5050 logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before InitGoogleLogging() is written to STDERR Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932 4475 main.cpp:243] Build: 2016-08-26 23:06:27 by centos Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128 4475 main.cpp:244] Version: 1.0.1 Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138 4475 main.cpp:247] Git tag: 1.0.1 Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144 4475 main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3 Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127 4475 logging.cpp:194] INFO level logging started! Sep 26 15:51:42 zk2 mesos-slave[4475]: mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string& Result::error() const [with T = std::basic_string; std::string = std::basic_string]: Assertion `isError()' failed. Sep 26 15:51:42 zk2 mesos-slave[4475]: *** Aborted at 1474876302 (unix time) try "date -d @1474876302" if you are using GNU date *** Sep 26 15:51:42 zk2 mesos-slave[4475]: PC: @ 0x7f795491e5f7 __GI_raise Sep 26 15:51:42 zk2 mesos-slave[4475]: *** SIGABRT (@0x117b) received by PID 4475 (TID 0x7f79577b98c0) from PID 4475; stack trace: *** Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f79551d8100 (unknown) Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f795491e5f7 __GI_raise Sep 26 15:51:42 zk2 mesos-slave[4475]:@ 0x7f795491fce8 __GI_abort Sep 26 15:51:42
[jira] [Comment Edited] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[
https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505567#comment-15505567
]
kasim edited comment on MESOS-6205 at 9/20/16 4:53 AM:
---
Thanks, empty work_dir works. But I don't understand how this situation happen.
At first, I started only one master and zookeeper for test.
{code}
$ cat /etc/mesos/zk
zk://10.142.55.190:2181/mesos
{code}
The slave on same machine was able to connect master, but other couldn't.
So I tried to start three mesos-master and zookeepers to consist cluster, and
change `/etc/mesos/zk` to
{code}
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
{code}, then got above error.
Is this mean I need clear wrok_dir everytime when adding a new mesos-master?
was (Author: mithril):
Thanks, empty work_dir works. But I don't understand how this situation happen.
At first, I started only one master and zookeeper for test.
{code}
$ cat /etc/mesos/zk
zk://10.142.55.190:2181/mesos
{code}
The slave on same machine was able to connect master, but other couldn't.
So I tried to start three master to consist cluster, change `/etc/mesos/zk` to
{code}
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
{code}, then got above error.
Is this mean I need clear wrok_dir everytime when adding a new mesos-master?
> mesos-master can not found mesos-slave, and elect a new leader in a short
> interval
> --
>
> Key: MESOS-6205
> URL: https://issues.apache.org/jira/browse/MESOS-6205
> Project: Mesos
> Issue Type: Bug
> Components: master
> Environment: ubuntu 12 x64, centos 6.5 x64, centos 7.2 x64
>Reporter: kasim
>
> I follow this
> [doc][https://open.mesosphere.com/getting-started/install/#verifying-installation]
> to setup mesos cluster.
> There are three vm(ubuntu 12, centos 6.5, centos 7.2).
> {code}
> $ cat /etc/hosts
> 10.142.55.190 zk1
> 10.142.55.196 zk2
> 10.142.55.202 zk3
> {code}
> config in each mathine:
> {code}
> $ cat /etc/mesos/zk
> zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
> {code}
>
> After start zookeeper, mesos-master and mesos-slave in three vm, I can view
> the mesos webui(10.142.55.190:5050), but agents count is 0.
> After a little time, mesos page get error:
> {code}
> Failed to connect to 10.142.55.190:5050!
> Retrying in 16 seconds...
> {code}
> (I found that zookeeper would elect a new leader in a short interval)
>
> mesos-master cmd:
> {code}
> mesos-master --agent_ping_timeout="15secs"
> --agent_reregister_timeout="10mins" --allocation_interval="1secs"
> --allocator="HierarchicalDRF" --authenticate_agents="false"
> --authenticate_frameworks="false" --authenticate_http_frameworks="false"
> --authenticate_http_readonly="false" --authenticate_http_readwrite="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --ip="10.142.55.190"
> --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --port="5050" --quiet="false" --quorum="2"
> --recovery_agent_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
> --registry_strict="false" --root_submissions="true" --user_sorter="drf"
> --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/var/lib/mesos"
> --zk="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos"
> {code}
> mesos-slave cmd:
> {code}
> mesos-slave --appc_simple_discovery_uri_prefix="http://;
> --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticatee="crammd5"
> --authentication_backoff_factor="1secs" --authorizer="local"
> --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
> --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins"
> --docker="docker" --docker_kill_orphans="true"
> --docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker"
> --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
> --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins"
>
[jira] [Comment Edited] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[
https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505567#comment-15505567
]
kasim edited comment on MESOS-6205 at 9/20/16 4:51 AM:
---
Thanks, empty work_dir works. But I don't understand how this situation happen.
At first, I started only one master and zookeeper for test.
{code}
$ cat /etc/mesos/zk
zk://10.142.55.190:2181/mesos
{code}
The slave on same machine was able to connect master, but other couldn't.
So I tried to start three master to consist cluster, change `/etc/mesos/zk` to
{code}
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
{code}, then got above error.
Is this mean I need clear wrok_dir everytime when adding a new mesos-master?
was (Author: mithril):
Thanks, empty work_dir work. But I don't understand how this situation happen.
At first, I started only one master and zookeeper for test.
{code}
$ cat /etc/mesos/zk
zk://10.142.55.190:2181/mesos
{code}
The slave on same machine was able to connect master, but other couldn't.
So I tried to start three master to consist cluster, change `/etc/mesos/zk` to
{code}
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
{code}, then got above error.
Is this mean I need clear wrok_dir everytime when adding a new mesos-master?
> mesos-master can not found mesos-slave, and elect a new leader in a short
> interval
> --
>
> Key: MESOS-6205
> URL: https://issues.apache.org/jira/browse/MESOS-6205
> Project: Mesos
> Issue Type: Bug
> Components: master
> Environment: ubuntu 12 x64, centos 6.5 x64, centos 7.2 x64
>Reporter: kasim
>
> I follow this
> [doc][https://open.mesosphere.com/getting-started/install/#verifying-installation]
> to setup mesos cluster.
> There are three vm(ubuntu 12, centos 6.5, centos 7.2).
> {code}
> $ cat /etc/hosts
> 10.142.55.190 zk1
> 10.142.55.196 zk2
> 10.142.55.202 zk3
> {code}
> config in each mathine:
> {code}
> $ cat /etc/mesos/zk
> zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
> {code}
>
> After start zookeeper, mesos-master and mesos-slave in three vm, I can view
> the mesos webui(10.142.55.190:5050), but agents count is 0.
> After a little time, mesos page get error:
> {code}
> Failed to connect to 10.142.55.190:5050!
> Retrying in 16 seconds...
> {code}
> (I found that zookeeper would elect a new leader in a short interval)
>
> mesos-master cmd:
> {code}
> mesos-master --agent_ping_timeout="15secs"
> --agent_reregister_timeout="10mins" --allocation_interval="1secs"
> --allocator="HierarchicalDRF" --authenticate_agents="false"
> --authenticate_frameworks="false" --authenticate_http_frameworks="false"
> --authenticate_http_readonly="false" --authenticate_http_readwrite="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --ip="10.142.55.190"
> --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --port="5050" --quiet="false" --quorum="2"
> --recovery_agent_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
> --registry_strict="false" --root_submissions="true" --user_sorter="drf"
> --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/var/lib/mesos"
> --zk="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos"
> {code}
> mesos-slave cmd:
> {code}
> mesos-slave --appc_simple_discovery_uri_prefix="http://;
> --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticatee="crammd5"
> --authentication_backoff_factor="1secs" --authorizer="local"
> --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
> --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins"
> --docker="docker" --docker_kill_orphans="true"
> --docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker"
> --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
> --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins"
> --executor_shutdown_grace_period="5secs"
>
[jira] [Commented] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[
https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505567#comment-15505567
]
kasim commented on MESOS-6205:
--
Thanks, empty work_dir work. But I don't understand how this situation happen.
At first, I started only one master and zookeeper for test.
{code}
$ cat /etc/mesos/zk
zk://10.142.55.190:2181/mesos
{code}
The slave on same machine was able to connect master, but other couldn't.
So I tried to start three master to consist cluster, change `/etc/mesos/zk` to
{code}
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
{code}, then got above error.
Is this mean I need clear wrok_dir everytime when adding a new mesos-master?
> mesos-master can not found mesos-slave, and elect a new leader in a short
> interval
> --
>
> Key: MESOS-6205
> URL: https://issues.apache.org/jira/browse/MESOS-6205
> Project: Mesos
> Issue Type: Bug
> Components: master
> Environment: ubuntu 12 x64, centos 6.5 x64, centos 7.2 x64
>Reporter: kasim
>
> I follow this
> [doc][https://open.mesosphere.com/getting-started/install/#verifying-installation]
> to setup mesos cluster.
> There are three vm(ubuntu 12, centos 6.5, centos 7.2).
> {code}
> $ cat /etc/hosts
> 10.142.55.190 zk1
> 10.142.55.196 zk2
> 10.142.55.202 zk3
> {code}
> config in each mathine:
> {code}
> $ cat /etc/mesos/zk
> zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
> {code}
>
> After start zookeeper, mesos-master and mesos-slave in three vm, I can view
> the mesos webui(10.142.55.190:5050), but agents count is 0.
> After a little time, mesos page get error:
> {code}
> Failed to connect to 10.142.55.190:5050!
> Retrying in 16 seconds...
> {code}
> (I found that zookeeper would elect a new leader in a short interval)
>
> mesos-master cmd:
> {code}
> mesos-master --agent_ping_timeout="15secs"
> --agent_reregister_timeout="10mins" --allocation_interval="1secs"
> --allocator="HierarchicalDRF" --authenticate_agents="false"
> --authenticate_frameworks="false" --authenticate_http_frameworks="false"
> --authenticate_http_readonly="false" --authenticate_http_readwrite="false"
> --authenticators="crammd5" --authorizers="local" --framework_sorter="drf"
> --help="false" --hostname_lookup="true" --http_authenticators="basic"
> --initialize_driver_logging="true" --ip="10.142.55.190"
> --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
> --logging_level="INFO" --max_agent_ping_timeouts="5"
> --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000"
> --port="5050" --quiet="false" --quorum="2"
> --recovery_agent_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
> --registry_strict="false" --root_submissions="true" --user_sorter="drf"
> --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/var/lib/mesos"
> --zk="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos"
> {code}
> mesos-slave cmd:
> {code}
> mesos-slave --appc_simple_discovery_uri_prefix="http://;
> --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false"
> --authenticate_http_readwrite="false" --authenticatee="crammd5"
> --authentication_backoff_factor="1secs" --authorizer="local"
> --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false"
> --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false"
> --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins"
> --docker="docker" --docker_kill_orphans="true"
> --docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker"
> --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume"
> --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins"
> --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname="10.142.55.190"
> --hostname_lookup="true" --http_authenticators="basic"
> --http_command_executor="false" --image_provisioner_backend="copy"
> --initialize_driver_logging="true" --ip="10.142.55.190"
> --isolation="posix/cpu,posix/mem" --launcher="posix"
> --launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos"
> --logbufsecs="0" --logging_level="INFO"
>
[jira] [Updated] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[ https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kasim updated MESOS-6205: - Description: I follow this [doc][https://open.mesosphere.com/getting-started/install/#verifying-installation] to setup mesos cluster. There are three vm(ubuntu 12, centos 6.5, centos 7.2). $ cat /etc/hosts 10.142.55.190 zk1 10.142.55.196 zk2 10.142.55.202 zk3 config in each mathine: $ cat /etc/mesos/zk zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos After start zookeeper, mesos-master and mesos-slave in three vm, I can view the mesos webui(10.142.55.190:5050), but agents count is 0. After a little time, mesos page get error: Failed to connect to 10.142.55.190:5050! Retrying in 16 seconds... (I found that zookeeper would elect a new leader in a short interval) mesos-master cmd: ``` mesos-master --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="false" --authenticate_frameworks="false" --authenticate_http_frameworks="false" --authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticators="crammd5" --authorizers="local" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --initialize_driver_logging="true" --ip="10.142.55.190" --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --port="5050" --quiet="false" --quorum="2" --recovery_agent_removal_limit="100%" --registry="replicated_log" --registry_fetch_timeout="1mins" --registry_store_timeout="20secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui" --work_dir="/var/lib/mesos" --zk="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos" ``` mesos-slave cmd: ``` mesos-slave --appc_simple_discovery_uri_prefix="http://; --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname="10.142.55.190" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --image_provisioner_backend="copy" --initialize_driver_logging="true" --ip="10.142.55.190" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" --master="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="1secs" --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/var/lib/mesos" ``` When I run mesos-master from command-line, I got ``` I0919 17:20:19.286264 17550 replica.cpp:673] Replica in VOTING status received a broadcasted recover request from (583)@10.142.55.202:5050 F0919 17:20:20.009371 17556 master.cpp:1536] Recovery failed: Failed to recover registrar: Failed to perform fetch within 1mins *** Check failure stack trace: *** @ 0x7f9db78458dd google::LogMessage::Fail() @ 0x7f9db784771d google::LogMessage::SendToLog() @ 0x7f9db78454cc google::LogMessage::Flush() @ 0x7f9db7848019 google::LogMessageFatal::~LogMessageFatal() @ 0x7f9db6e2dbbc mesos::internal::master::fail() @ 0x7f9db6e75b20
[jira] [Updated] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[ https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kasim updated MESOS-6205: - Description: I follow this [doc][https://open.mesosphere.com/getting-started/install/#verifying-installation] to setup mesos cluster. There are three vm(ubuntu 12, centos 6.5, centos 7.2). $ cat /etc/hosts 10.142.55.190 zk1 10.142.55.196 zk2 10.142.55.202 zk3 config in each mathine: $ cat /etc/mesos/zk zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos After start zookeeper, mesos-master and mesos-slave in three vm, I can view the mesos webui(10.142.55.190:5050), but agents count is 0. After a little time, mesos page get error: Failed to connect to 10.142.55.190:5050! Retrying in 16 seconds... (I found that zookeeper would elect a new leader in a short interval) mesos-master cmd: ``` mesos-master --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="false" --authenticate_frameworks="false" --authenticate_http_frameworks="false" --authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticators="crammd5" --authorizers="local" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --initialize_driver_logging="true" --ip="10.142.55.190" --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --port="5050" --quiet="false" --quorum="2" --recovery_agent_removal_limit="100%" --registry="replicated_log" --registry_fetch_timeout="1mins" --registry_store_timeout="20secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui" --work_dir="/var/lib/mesos" --zk="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos" ``` mesos-slave cmd: ``` mesos-slave --appc_simple_discovery_uri_prefix="http://; --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname="10.142.55.190" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --image_provisioner_backend="copy" --initialize_driver_logging="true" --ip="10.142.55.190" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" --master="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="1secs" --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/var/lib/mesos" ``` When I run mesos-master from command-line, I got ``` I0919 17:20:19.286264 17550 replica.cpp:673] Replica in VOTING status received a broadcasted recover request from (583)@10.142.55.202:5050 F0919 17:20:20.009371 17556 master.cpp:1536] Recovery failed: Failed to recover registrar: Failed to perform fetch within 1mins *** Check failure stack trace: *** @ 0x7f9db78458dd google::LogMessage::Fail() @ 0x7f9db784771d google::LogMessage::SendToLog() @ 0x7f9db78454cc google::LogMessage::Flush() @ 0x7f9db7848019 google::LogMessageFatal::~LogMessageFatal() @ 0x7f9db6e2dbbc mesos::internal::master::fail() @ 0x7f9db6e75b20
[jira] [Updated] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[ https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kasim updated MESOS-6205: - Description: I follow this [doc][https://open.mesosphere.com/getting-started/install/#verifying-installation] to setup mesos cluster. There are three vm(ubuntu 12, centos 6.5, centos 7.2). $ cat /etc/hosts 10.142.55.190 zk1 10.142.55.196 zk2 10.142.55.202 zk3 config in each mathine: $ cat /etc/mesos/zk zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos After start zookeeper, mesos-master and mesos-slave in three vm, I can view the mesos webui(10.142.55.190:5050), but agents count is 0. After a little time, mesos page get error: Failed to connect to 10.142.55.190:5050! Retrying in 16 seconds... (I found that zookeeper would elect a new leader in a short interval) mesos-master cmd: ``` mesos-master --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="false" --authenticate_frameworks="false" --authenticate_http_frameworks="false" --authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticators="crammd5" --authorizers="local" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --initialize_driver_logging="true" --ip="10.142.55.190" --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --port="5050" --quiet="false" --quorum="2" --recovery_agent_removal_limit="100%" --registry="replicated_log" --registry_fetch_timeout="1mins" --registry_store_timeout="20secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui" --work_dir="/var/lib/mesos" --zk="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos" ``` mesos-slave cmd: ``` mesos-slave --appc_simple_discovery_uri_prefix="http://; --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname="10.142.55.190" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --image_provisioner_backend="copy" --initialize_driver_logging="true" --ip="10.142.55.190" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" --master="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="1secs" --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/var/lib/mesos" ``` When I run mesos-master from command-line, I got ``` I0919 17:20:19.286264 17550 replica.cpp:673] Replica in VOTING status received a broadcasted recover request from (583)@10.142.55.202:5050 F0919 17:20:20.009371 17556 master.cpp:1536] Recovery failed: Failed to recover registrar: Failed to perform fetch within 1mins *** Check failure stack trace: *** @ 0x7f9db78458dd google::LogMessage::Fail() @ 0x7f9db784771d google::LogMessage::SendToLog() @ 0x7f9db78454cc google::LogMessage::Flush() @ 0x7f9db7848019 google::LogMessageFatal::~LogMessageFatal() @ 0x7f9db6e2dbbc mesos::internal::master::fail() @ 0x7f9db6e75b20
[jira] [Updated] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[ https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kasim updated MESOS-6205: - Description: I follow this [doc][https://open.mesosphere.com/getting-started/install/#verifying-installation] to setup mesos cluster. There are three vm(ubuntu 12, centos 6.5, centos 7.2). $ cat /etc/hosts 10.142.55.190 zk1 10.142.55.196 zk2 10.142.55.202 zk3 config in each mathine: $ cat /etc/mesos/zk zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos After start zookeeper, mesos-master and mesos-slave in three vm, I can view the mesos webui(10.142.55.190:5050), but agents count is 0. After a little time, mesos page get error: Failed to connect to 10.142.55.190:5050! Retrying in 16 seconds... (I found that zookeeper would elect a new leader in a short interval) mesos-master cmd: ``` mesos-master --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="false" --authenticate_frameworks="false" --authenticate_http_frameworks="false" --authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticators="crammd5" --authorizers="local" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --initialize_driver_logging="true" --ip="10.142.55.190" --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --port="5050" --quiet="false" --quorum="2" --recovery_agent_removal_limit="100%" --registry="replicated_log" --registry_fetch_timeout="1mins" --registry_store_timeout="20secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui" --work_dir="/var/lib/mesos" --zk="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos" ``` mesos-slave cmd: ``` mesos-slave --appc_simple_discovery_uri_prefix="http://; --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="false" --authenticate_http_readwrite="false" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io; --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname="10.142.55.190" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --image_provisioner_backend="copy" --initialize_driver_logging="true" --ip="10.142.55.190" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/usr/libexec/mesos" --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO" --master="zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="1secs" --revocable_cpu_low_priority="true" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/var/lib/mesos" ``` When I run mesos-master from command-line, I got ``` I0919 17:20:19.286264 17550 replica.cpp:673] Replica in VOTING status received a broadcasted recover request from (583)@10.142.55.202:5050 F0919 17:20:20.009371 17556 master.cpp:1536] Recovery failed: Failed to recover registrar: Failed to perform fetch within 1mins *** Check failure stack trace: *** @ 0x7f9db78458dd google::LogMessage::Fail() @ 0x7f9db784771d google::LogMessage::SendToLog() @ 0x7f9db78454cc google::LogMessage::Flush() @ 0x7f9db7848019 google::LogMessageFatal::~LogMessageFatal() @ 0x7f9db6e2dbbc mesos::internal::master::fail() @ 0x7f9db6e75b20
[jira] [Updated] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[
https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kasim updated MESOS-6205:
-
Description:
I follow this
[doc][https://open.mesosphere.com/getting-started/install/#verifying-installation]
to setup mesos cluster.
There are three vm(ubuntu 12, centos 6.5, centos 7.2).
$ cat /etc/hosts
10.142.55.190 zk1
10.142.55.196 zk2
10.142.55.202 zk3
config in each mathine:
$ cat /etc/mesos/zk
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
After start zookeeper, mesos-master and mesos-slave in three vm, I can view the
mesos webui(10.142.55.190:5050), but agents count is 0.
After a little time, mesos page get error:
Failed to connect to 10.142.55.190:5050!
Retrying in 16 seconds...
(I found that zookeeper would elect a new leader in a short interval)
master info log:
I0919 15:54:59.677438 13281 http.cpp:2022] Redirecting request for
/master/state?jsonp=angular.callbacks._1x to the leading master zk3
I0919 15:55:00.098667 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (768)@10.142.55.202:5050
I0919 15:55:00.385279 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (185)@10.142.55.196:5050
I0919 15:55:00.79 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (771)@10.142.55.202:5050
I0919 15:55:01.347291 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (188)@10.142.55.196:5050
I0919 15:55:01.597682 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (774)@10.142.55.202:5050
I0919 15:55:02.257159 13282 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (191)@10.142.55.196:5050
I0919 15:55:02.370692 13287 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (777)@10.142.55.202:5050
I0919 15:55:03.205920 13285 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (780)@10.142.55.202:5050
I0919 15:55:03.260007 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (194)@10.142.55.196:5050
I0919 15:55:03.929611 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (783)@10.142.55.202:5050
I0919 15:55:04.033308 13287 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (197)@10.142.55.196:5050
I0919 15:55:04.591275 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (200)@10.142.55.196:5050
I0919 15:55:04.608211 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (786)@10.142.55.202:5050
I0919 15:55:05.184682 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (789)@10.142.55.202:5050
I0919 15:55:05.268277 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (203)@10.142.55.196:5050
I0919 15:55:05.775377 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (206)@10.142.55.196:5050
I0919 15:55:05.916445 13285 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (792)@10.142.55.202:5050
I0919 15:55:06.744927 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (209)@10.142.55.196:5050
I0919 15:55:07.378521 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (5)@10.142.55.202:5050
I0919 15:55:07.393311 13285 network.hpp:430] ZooKeeper group memberships
changed
I0919 15:55:07.393427 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000709' in ZooKeeper
I0919 15:55:07.393985 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000711' in ZooKeeper
I0919 15:55:07.394394 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000714' in ZooKeeper
I0919 15:55:07.394843 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000715' in ZooKeeper
I0919 15:55:07.395418 13285 network.hpp:478] ZooKeeper group PIDs: {
log-replica(1)@10.142.55.190:5050, log-replica(1)@10.142.55.196:5050,
log-replica(1)@10.142.55.202:5050 }
I0919 15:55:08.178272 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (14)@10.142.55.202:5050
I0919 15:55:09.059562 13282 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (21)@10.142.55.202:5050
I0919 15:55:09.700711 13286 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (24)@10.142.55.202:5050
I0919 15:55:09.742185 13287 http.cpp:381] HTTP GET for
[jira] [Updated] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[
https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kasim updated MESOS-6205:
-
Description:
I follow this
[doc][https://open.mesosphere.com/getting-started/install/#verifying-installation]
to setup mesos cluster.
There are three vm(ubuntu 12, centos 6.5, centos 7.2).
$ cat /etc/hosts
10.142.55.190 zk1
10.142.55.196 zk2
10.142.55.202 zk3
config in each mathine:
$ cat /etc/mesos/zk
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
After I start zookeeper, mesos-master and mesos-slave in three vm, I can view
the mesos webui(10.142.55.190:5050). I found agents count is 0.
After a little time, mesos page get error:
Failed to connect to 10.142.55.190:5050!
Retrying in 16 seconds...
(I found that zookeeper would elect a new leader in a short interval)
master info log:
I0919 15:54:59.677438 13281 http.cpp:2022] Redirecting request for
/master/state?jsonp=angular.callbacks._1x to the leading master zk3
I0919 15:55:00.098667 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (768)@10.142.55.202:5050
I0919 15:55:00.385279 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (185)@10.142.55.196:5050
I0919 15:55:00.79 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (771)@10.142.55.202:5050
I0919 15:55:01.347291 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (188)@10.142.55.196:5050
I0919 15:55:01.597682 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (774)@10.142.55.202:5050
I0919 15:55:02.257159 13282 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (191)@10.142.55.196:5050
I0919 15:55:02.370692 13287 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (777)@10.142.55.202:5050
I0919 15:55:03.205920 13285 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (780)@10.142.55.202:5050
I0919 15:55:03.260007 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (194)@10.142.55.196:5050
I0919 15:55:03.929611 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (783)@10.142.55.202:5050
I0919 15:55:04.033308 13287 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (197)@10.142.55.196:5050
I0919 15:55:04.591275 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (200)@10.142.55.196:5050
I0919 15:55:04.608211 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (786)@10.142.55.202:5050
I0919 15:55:05.184682 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (789)@10.142.55.202:5050
I0919 15:55:05.268277 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (203)@10.142.55.196:5050
I0919 15:55:05.775377 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (206)@10.142.55.196:5050
I0919 15:55:05.916445 13285 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (792)@10.142.55.202:5050
I0919 15:55:06.744927 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (209)@10.142.55.196:5050
I0919 15:55:07.378521 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (5)@10.142.55.202:5050
I0919 15:55:07.393311 13285 network.hpp:430] ZooKeeper group memberships
changed
I0919 15:55:07.393427 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000709' in ZooKeeper
I0919 15:55:07.393985 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000711' in ZooKeeper
I0919 15:55:07.394394 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000714' in ZooKeeper
I0919 15:55:07.394843 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000715' in ZooKeeper
I0919 15:55:07.395418 13285 network.hpp:478] ZooKeeper group PIDs: {
log-replica(1)@10.142.55.190:5050, log-replica(1)@10.142.55.196:5050,
log-replica(1)@10.142.55.202:5050 }
I0919 15:55:08.178272 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (14)@10.142.55.202:5050
I0919 15:55:09.059562 13282 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (21)@10.142.55.202:5050
I0919 15:55:09.700711 13286 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (24)@10.142.55.202:5050
I0919 15:55:09.742185 13287 http.cpp:381] HTTP GET
[jira] [Updated] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
[
https://issues.apache.org/jira/browse/MESOS-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kasim updated MESOS-6205:
-
Description:
I follow this
[doc][https://open.mesosphere.com/getting-started/install/#verifying-installation]
to setup mesos cluster.
There are three vm(ubuntu 12, centos 6.5, centos 7.2).
$ cat /etc/hosts
10.142.55.190 zk1
10.142.55.196 zk2
10.142.55.202 zk3
config in each mathine:
$ cat /etc/mesos/zk
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
After I start zookeeper, mesos-master and mesos-slave in three vm, I can view
the mesos webui(10.142.55.190:5050). I found agents count is 0.
After a little time, mesos page get error:
Failed to connect to 10.142.55.190:5050!
Retrying in 16 seconds...
(I found that zookeeper would elect a new leader in a short interval)
master info log:
I0919 15:54:59.677438 13281 http.cpp:2022] Redirecting request for
/master/state?jsonp=angular.callbacks._1x to the leading master zk3
I0919 15:55:00.098667 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (768)@10.142.55.202:5050
I0919 15:55:00.385279 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (185)@10.142.55.196:5050
I0919 15:55:00.79 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (771)@10.142.55.202:5050
I0919 15:55:01.347291 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (188)@10.142.55.196:5050
I0919 15:55:01.597682 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (774)@10.142.55.202:5050
I0919 15:55:02.257159 13282 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (191)@10.142.55.196:5050
I0919 15:55:02.370692 13287 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (777)@10.142.55.202:5050
I0919 15:55:03.205920 13285 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (780)@10.142.55.202:5050
I0919 15:55:03.260007 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (194)@10.142.55.196:5050
I0919 15:55:03.929611 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (783)@10.142.55.202:5050
I0919 15:55:04.033308 13287 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (197)@10.142.55.196:5050
I0919 15:55:04.591275 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (200)@10.142.55.196:5050
I0919 15:55:04.608211 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (786)@10.142.55.202:5050
I0919 15:55:05.184682 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (789)@10.142.55.202:5050
I0919 15:55:05.268277 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (203)@10.142.55.196:5050
I0919 15:55:05.775377 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (206)@10.142.55.196:5050
I0919 15:55:05.916445 13285 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (792)@10.142.55.202:5050
I0919 15:55:06.744927 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (209)@10.142.55.196:5050
I0919 15:55:07.378521 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (5)@10.142.55.202:5050
I0919 15:55:07.393311 13285 network.hpp:430] ZooKeeper group memberships
changed
I0919 15:55:07.393427 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000709' in ZooKeeper
I0919 15:55:07.393985 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000711' in ZooKeeper
I0919 15:55:07.394394 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000714' in ZooKeeper
I0919 15:55:07.394843 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000715' in ZooKeeper
I0919 15:55:07.395418 13285 network.hpp:478] ZooKeeper group PIDs: {
log-replica(1)@10.142.55.190:5050, log-replica(1)@10.142.55.196:5050,
log-replica(1)@10.142.55.202:5050 }
I0919 15:55:08.178272 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (14)@10.142.55.202:5050
I0919 15:55:09.059562 13282 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (21)@10.142.55.202:5050
I0919 15:55:09.700711 13286 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (24)@10.142.55.202:5050
I0919 15:55:09.742185 13287 http.cpp:381] HTTP GET
[jira] [Created] (MESOS-6205) mesos-master can not found mesos-slave, and elect a new leader in a short interval
kasim created MESOS-6205:
Summary: mesos-master can not found mesos-slave, and elect a new
leader in a short interval
Key: MESOS-6205
URL: https://issues.apache.org/jira/browse/MESOS-6205
Project: Mesos
Issue Type: Bug
Components: master
Environment: ubuntu 12 x64, centos 6.5 x64, centos 7.2 x64
Reporter: kasim
I follow this [doc][1] to setup mesos cluster.
There are three vm(ubuntu 12, centos 6.5, centos 7.2).
$ cat /etc/hosts
10.142.55.190 zk1
10.142.55.196 zk2
10.142.55.202 zk3
config in each mathine:
$ cat /etc/mesos/zk
zk://10.142.55.190:2181,10.142.55.196:2181,10.142.55.202:2181/mesos
After I start mesos-master in three vm, I can view the mesos
webui(10.142.55.190:5050), but after a little time, mesos page get error:
Failed to connect to 10.142.55.190:5050!
Retrying in 16 seconds...
(I found that zookeeper would elect a new leader in a short interval)
master info log:
I0919 15:54:59.677438 13281 http.cpp:2022] Redirecting request for
/master/state?jsonp=angular.callbacks._1x to the leading master zk3
I0919 15:55:00.098667 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (768)@10.142.55.202:5050
I0919 15:55:00.385279 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (185)@10.142.55.196:5050
I0919 15:55:00.79 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (771)@10.142.55.202:5050
I0919 15:55:01.347291 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (188)@10.142.55.196:5050
I0919 15:55:01.597682 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (774)@10.142.55.202:5050
I0919 15:55:02.257159 13282 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (191)@10.142.55.196:5050
I0919 15:55:02.370692 13287 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (777)@10.142.55.202:5050
I0919 15:55:03.205920 13285 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (780)@10.142.55.202:5050
I0919 15:55:03.260007 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (194)@10.142.55.196:5050
I0919 15:55:03.929611 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (783)@10.142.55.202:5050
I0919 15:55:04.033308 13287 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (197)@10.142.55.196:5050
I0919 15:55:04.591275 13284 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (200)@10.142.55.196:5050
I0919 15:55:04.608211 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (786)@10.142.55.202:5050
I0919 15:55:05.184682 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (789)@10.142.55.202:5050
I0919 15:55:05.268277 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (203)@10.142.55.196:5050
I0919 15:55:05.775377 13281 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (206)@10.142.55.196:5050
I0919 15:55:05.916445 13285 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (792)@10.142.55.202:5050
I0919 15:55:06.744927 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (209)@10.142.55.196:5050
I0919 15:55:07.378521 13283 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (5)@10.142.55.202:5050
I0919 15:55:07.393311 13285 network.hpp:430] ZooKeeper group memberships
changed
I0919 15:55:07.393427 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000709' in ZooKeeper
I0919 15:55:07.393985 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000711' in ZooKeeper
I0919 15:55:07.394394 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000714' in ZooKeeper
I0919 15:55:07.394843 13285 group.cpp:706] Trying to get
'/mesos/log_replicas/000715' in ZooKeeper
I0919 15:55:07.395418 13285 network.hpp:478] ZooKeeper group PIDs: {
log-replica(1)@10.142.55.190:5050, log-replica(1)@10.142.55.196:5050,
log-replica(1)@10.142.55.202:5050 }
I0919 15:55:08.178272 13280 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (14)@10.142.55.202:5050
I0919 15:55:09.059562 13282 replica.cpp:673] Replica in VOTING status
received a broadcasted recover request from (21)@10.142.55.202:5050
I0919 15:55:09.700711 13286 replica.cpp:673] Replica in VOTING status
