seems ok to me. But I wasnt really clear on frameworkUser , and
frameworkSuperUser - both have necessary permissions. I am also trying
out the dockerizerd resource manager from Brandon. Apologies for the
verbosity...
myriad-config-default.xml
mesosMaster: 192.168.1.110:5050
checkpoint: false
frameworkFailoverTimeout: 43200000
frameworkName: MyriadAlpha
frameworkRole:
frameworkUser: ubuntu
frameworkSuperUser: root
nativeLibrary: /usr/local/lib/libmesos.so
zkServers: localhost:2181
zkTimeout: 20000
restApiPort: 8192
resource manager logs :
docker run --net=host -e HADOOP_VER=2.7.1 -e HADOOP_USER=ubuntu -e
HADOOP_NAMENODE=192.168.1.110 -t -v
~/Dev/proto/incubator-myriad/docker/config:/myriad-conf
prabhuinbarajan/myriad
-----------------
15/12/04 19:17:21 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 997
15/12/04 19:17:24 INFO resourcemanager.ClientRMService: Allocated new
applicationId: 1
15/12/04 19:17:24 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:24 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 999
15/12/04 19:17:27 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:27 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 997
15/12/04 19:17:29 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:29 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 999
15/12/04 19:17:32 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:32 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 997
15/12/04 19:17:34 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:34 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 999
15/12/04 19:17:37 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:37 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 997
15/12/04 19:17:39 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:39 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 999
15/12/04 19:17:39 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 999
15/12/04 19:17:40 INFO resourcemanager.ClientRMService: Application with id
1 submitted by user ubuntu
15/12/04 19:17:40 INFO rmapp.RMAppImpl: Storing application with id
application_1449255931427_0001
15/12/04 19:17:40 INFO resourcemanager.RMAuditLogger: USER=ubuntu
IP=192.168.1.33 OPERATION=Submit Application Request
TARGET=ClientRMService RESULT=SUCCESS
APPID=application_1449255931427_0001
15/12/04 19:17:40 INFO rmapp.RMAppImpl: application_1449255931427_0001
State change from NEW to NEW_SAVING
15/12/04 19:17:40 INFO recovery.RMStateStore: Storing info for app:
application_1449255931427_0001
15/12/04 19:17:40 INFO rmapp.RMAppImpl: application_1449255931427_0001
State change from NEW_SAVING to SUBMITTED
15/12/04 19:17:40 WARN security.UserGroupInformation: No groups available
for user ubuntu
15/12/04 19:17:40 INFO fair.FairScheduler: Accepted application
application_1449255931427_0001 from user: ubuntu, in queue: default,
currently num of applications: 1
15/12/04 19:17:40 INFO rmapp.RMAppImpl: application_1449255931427_0001
State change from SUBMITTED to ACCEPTED
15/12/04 19:17:40 INFO resourcemanager.ApplicationMasterService:
Registering app attempt : appattempt_1449255931427_0001_000001
15/12/04 19:17:40 INFO attempt.RMAppAttemptImpl:
appattempt_1449255931427_0001_000001 State change from NEW to SUBMITTED
15/12/04 19:17:40 INFO fair.FairScheduler: Added Application Attempt
appattempt_1449255931427_0001_000001 to scheduler from user: ubuntu
15/12/04 19:17:40 INFO attempt.RMAppAttemptImpl:
appattempt_1449255931427_0001_000001 State change from SUBMITTED to
SCHEDULED
15/12/04 19:17:42 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:42 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 997
15/12/04 19:17:44 INFO handlers.ResourceOffersEventHandler: Received offers
1
15/12/04 19:17:44 INFO handlers.ResourceOffersEventHandler: Offer not
sufficient for task with, cpu: 4.4, memory: 5504.0, ports: 999
15/12/04 19:17:47 INFO handlers.ResourceOffersEventHandler: Received offers
1
mesos slave 1 logs:
------------------------
ubuntu@ubuntu1:/var/log/mesos$ tail -f mesos-slave.{ERROR,INFO,WARNING}
==> mesos-slave.INFO <==
W1204 11:19:25.484510 24660 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:28.495157 24654 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
I1204 11:19:34.253932 24655 slave.cpp:3926] Current disk usage 36.45%. Max
allowed age: 3.748370011361713days
I1204 11:19:40.139240 24656 http.cpp:189] HTTP GET for /slave(1)/state from
192.168.1.3:62644 with User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X
10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73
Safari/537.36'
W1204 11:19:40.169255 24658 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:40.683851 24654 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
I1204 11:20:34.254957 24655 slave.cpp:3926] Current disk usage 36.45%. Max
allowed age: 3.748366616183600days
I1204 11:21:34.255574 24657 slave.cpp:3926] Current disk usage 36.45%. Max
allowed age: 3.748362372210949days
I1204 11:22:34.256463 24659 slave.cpp:3926] Current disk usage 36.45%. Max
allowed age: 3.748358977032824days
I1204 11:23:34.257474 24659 slave.cpp:3926] Current disk usage 36.45%. Max
allowed age: 3.748355581854711days
==> mesos-slave.WARNING <==
W1204 11:19:16.630030 24655 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:17.130801 24656 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:19.390177 24658 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:19.904623 24657 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:22.916709 24656 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:24.949373 24661 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:25.484510 24660 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:28.495157 24654 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:40.169255 24658 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
W1204 11:19:40.683851 24654 slave.cpp:4505] Failed to get resource
statistics for executor
'myriad_executor5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-00015958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-O55925958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0'
of framework 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001: Unknown container:
d69de4bd-d696-44a8-8a1f-a1909515ef40
mesos - slave 2 logs :
------------------------------
ubuntu@ubuntu2:/var/log/mesos$ tail -f mesos-slave.{INFO,ERROR,WARNING}
==> mesos-slave.INFO <==
I1204 11:18:46.767786 27777 http.cpp:189] HTTP GET for /slave(1)/state from
192.168.1.3:62649 with User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X
10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73
Safari/537.36'
I1204 11:18:50.083156 27775 http.cpp:189] HTTP GET for /slave(1)/state from
192.168.1.3:62649 with User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X
10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73
Safari/537.36'
I1204 11:18:51.238821 27773 http.cpp:189] HTTP GET for /slave(1)/state from
192.168.1.3:62649 with User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X
10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73
Safari/537.36'
I1204 11:18:58.242902 27778 slave.cpp:3926] Current disk usage 1.47%. Max
allowed age: 6.196790774951609days
I1204 11:19:58.244071 27779 slave.cpp:3926] Current disk usage 1.47%. Max
allowed age: 6.196764178349270days
I1204 11:20:58.244828 27778 slave.cpp:3926] Current disk usage 1.48%. Max
allowed age: 6.196737405610486days
I1204 11:21:58.246207 27777 slave.cpp:3926] Current disk usage 1.48%. Max
allowed age: 6.196710949917303days
I1204 11:22:58.247319 27777 slave.cpp:3926] Current disk usage 1.48%. Max
allowed age: 6.196684494224109days
I1204 11:23:58.248172 27775 slave.cpp:3926] Current disk usage 1.48%. Max
allowed age: 6.196657791939908days
I1204 11:24:58.249053 27776 slave.cpp:3926] Current disk usage 1.48%. Max
allowed age: 6.196631089655694days
mesos master logs:
--------------------------
ubuntu@master:/var/log/mesos$ tail -f mesos-master.{ERROR,INFO,WARNING}
==> mesos-master.INFO <==
I1204 11:21:06.762050 8915 master.cpp:4967] Sending 1 offers to framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000 (MyriadAlpha) at
[email protected]:59514
I1204 11:21:06.768584 8914 master.cpp:2918] Processing ACCEPT call for
offers: [ 1669be41-1157-4e33-a374-7d1e30674eb3-O478 ] on slave
5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 at slave(1)@192.168.1.33:5051
(ubuntu1.dev) for framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
(MyriadAlpha) at
[email protected]:59514
I1204 11:21:06.768824 8914 hierarchical.hpp:1103] Recovered cpus(*):3.8;
mem(*):1694.6; disk(*):27094; ports(*):[31000-31111, 31113-31443,
31445-31816, 31818-31875, 31877-32000] (total: cpus(*):8; mem(*):6917;
disk(*):27094; ports(*):[31000-32000], allocated: cpus(*):4.2;
mem(*):5222.4; ports(*):[31112-31112, 31444-31444, 31817-31817,
31876-31876]) on slave 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 from
framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
I1204 11:21:10.771049 8917 master.cpp:4967] Sending 1 offers to framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000 (MyriadAlpha) at
[email protected]:59514
I1204 11:21:10.775462 8916 master.cpp:2918] Processing ACCEPT call for
offers: [ 1669be41-1157-4e33-a374-7d1e30674eb3-O479 ] on slave
2c87707a-156a-4edc-908e-9f90761b32f2-S0 at slave(1)@192.168.1.34:5051
(192.168.1.34) for framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
(MyriadAlpha) at
[email protected]:59514
I1204 11:21:10.779511 8915 hierarchical.hpp:1103] Recovered cpus(*):3.7;
mem(*):6660; disk(*):771089; ports(*):[31000-31563, 31565-31926,
31928-32000] (total: cpus(*):4; mem(*):6708; disk(*):771089;
ports(*):[31000-32000], allocated: cpus(*):0.3; mem(*):48;
ports(*):[31564-31564, 31927-31927]) on slave
2c87707a-156a-4edc-908e-9f90761b32f2-S0 from framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000
I1204 11:21:11.773802 8912 master.cpp:4967] Sending 1 offers to framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000 (MyriadAlpha) at
[email protected]:59514
I1204 11:21:11.781312 8913 master.cpp:2918] Processing ACCEPT call for
offers: [ 1669be41-1157-4e33-a374-7d1e30674eb3-O480 ] on slave
5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 at slave(1)@192.168.1.33:5051
(ubuntu1.dev) for framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
(MyriadAlpha) at
[email protected]:59514
I1204 11:21:11.781577 8913 hierarchical.hpp:1103] Recovered cpus(*):3.8;
mem(*):1694.6; disk(*):27094; ports(*):[31000-31111, 31113-31443,
31445-31816, 31818-31875, 31877-32000] (total: cpus(*):8; mem(*):6917;
disk(*):27094; ports(*):[31000-32000], allocated: cpus(*):4.2;
mem(*):5222.4; ports(*):[31112-31112, 31444-31444, 31817-31817,
31876-31876]) on slave 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 from
framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
I1204 11:21:14.238896 8914 http.cpp:336] HTTP GET for /master/state.json
from 192.168.1.3:62643 with User-Agent='Mozilla/5.0 (Macintosh; Intel Mac
OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73
Safari/537.36'
==> mesos-master.WARNING <==
W1204 11:18:49.456564 8917 master.cpp:4408] Ignoring status update
TASK_LOST (UUID: 7dd56987-06a8-472e-9dfb-49b482d994db) for task
nm.medium.3410a508-4345-49ca-b658-ac3438202d82 of framework
5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-0001 from slave
5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 at slave(1)@192.168.1.33:5051
(ubuntu1.dev) because the framework is unknown
==> mesos-master.INFO <==
I1204 11:21:15.794827 8911 master.cpp:4967] Sending 1 offers to framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000 (MyriadAlpha) at
[email protected]:59514
I1204 11:21:15.806252 8917 master.cpp:2918] Processing ACCEPT call for
offers: [ 1669be41-1157-4e33-a374-7d1e30674eb3-O481 ] on slave
2c87707a-156a-4edc-908e-9f90761b32f2-S0 at slave(1)@192.168.1.34:5051
(192.168.1.34) for framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
(MyriadAlpha) at
[email protected]:59514
I1204 11:21:15.806967 8917 hierarchical.hpp:1103] Recovered cpus(*):3.7;
mem(*):6660; disk(*):771089; ports(*):[31000-31563, 31565-31926,
31928-32000] (total: cpus(*):4; mem(*):6708; disk(*):771089;
ports(*):[31000-32000], allocated: cpus(*):0.3; mem(*):48;
ports(*):[31564-31564, 31927-31927]) on slave
2c87707a-156a-4edc-908e-9f90761b32f2-S0 from framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000
I1204 11:21:16.804147 8914 master.cpp:4967] Sending 1 offers to framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000 (MyriadAlpha) at
[email protected]:59514
I1204 11:21:16.811523 8918 master.cpp:2918] Processing ACCEPT call for
offers: [ 1669be41-1157-4e33-a374-7d1e30674eb3-O482 ] on slave
5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 at slave(1)@192.168.1.33:5051
(ubuntu1.dev) for framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
(MyriadAlpha) at
[email protected]:59514
I1204 11:21:16.811856 8918 hierarchical.hpp:1103] Recovered cpus(*):3.8;
mem(*):1694.6; disk(*):27094; ports(*):[31000-31111, 31113-31443,
31445-31816, 31818-31875, 31877-32000] (total: cpus(*):8; mem(*):6917;
disk(*):27094; ports(*):[31000-32000], allocated: cpus(*):4.2;
mem(*):5222.4; ports(*):[31112-31112, 31444-31444, 31817-31817,
31876-31876]) on slave 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 from
framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
I1204 11:21:20.824043 8911 master.cpp:4967] Sending 1 offers to framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000 (MyriadAlpha) at
[email protected]:59514
I1204 11:21:20.833861 8917 master.cpp:2918] Processing ACCEPT call for
offers: [ 1669be41-1157-4e33-a374-7d1e30674eb3-O483 ] on slave
2c87707a-156a-4edc-908e-9f90761b32f2-S0 at slave(1)@192.168.1.34:5051
(192.168.1.34) for framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
(MyriadAlpha) at
[email protected]:59514
I1204 11:21:20.835531 8913 hierarchical.hpp:1103] Recovered cpus(*):3.7;
mem(*):6660; disk(*):771089; ports(*):[31000-31563, 31565-31926,
31928-32000] (total: cpus(*):4; mem(*):6708; disk(*):771089;
ports(*):[31000-32000], allocated: cpus(*):0.3; mem(*):48;
ports(*):[31564-31564, 31927-31927]) on slave
2c87707a-156a-4edc-908e-9f90761b32f2-S0 from framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000
I1204 11:21:21.826283 8911 master.cpp:4967] Sending 1 offers to framework
1669be41-1157-4e33-a374-7d1e30674eb3-0000 (MyriadAlpha) at
[email protected]:59514
I1204 11:21:21.835597 8916 master.cpp:2918] Processing ACCEPT call for
offers: [ 1669be41-1157-4e33-a374-7d1e30674eb3-O484 ] on slave
5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 at slave(1)@192.168.1.33:5051
(ubuntu1.dev) for framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
(MyriadAlpha) at
[email protected]:59514
I1204 11:21:21.837003 8916 hierarchical.hpp:1103] Recovered cpus(*):3.8;
mem(*):1694.6; disk(*):27094; ports(*):[31000-31111, 31113-31443,
31445-31816, 31818-31875, 31877-32000] (total: cpus(*):8; mem(*):6917;
disk(*):27094; ports(*):[31000-32000], allocated: cpus(*):4.2;
mem(*):5222.4; ports(*):[31112-31112, 31444-31444, 31817-31817,
31876-31876]) on slave 5958fb41-3d1c-4d8d-8d00-d82a0aa6d9e1-S0 from
framework 1669be41-1157-4e33-a374-7d1e30674eb3-0000
myriad executor logs:
----------------------------
ABORT: (/tmp/mesos-build/mesos-repo/3rdparty/libprocess/src/subprocess.cpp:177):
Failed to os::execvpe in childMain: Argument list too long*** Aborted
at 1449249974 (unix time) try "date -d @1449249974" if you are using
GNU date ***
PC: @ 0x7fbfd2c66cc9 (unknown)
*** SIGABRT (@0x5893) received by PID 22675 (TID 0x7fbfcbc4f700) from
PID 22675; stack trace: ***
@ 0x7fbfd3005340 (unknown)
@ 0x7fbfd2c66cc9 (unknown)
@ 0x7fbfd2c6a0d8 (unknown)
@ 0x40a902 _Abort()
@ 0x40a93c _Abort()
@ 0x7fbfd477ac3b process::childMain()
@ 0x7fbfd477cc6d std::_Function_handler<>::_M_invoke()
@ 0x7fbfd2d2a47d (unknown)
On Fri, Dec 4, 2015 at 11:09 AM, Sarjeet Singh <[email protected]>
wrote:
> Prabhu,
>
> Can you paste/send your mesos-slave, mesos-master log file, if this is OK?
>
> P.S., We might have seen this when frameworkUser was not set correctly in
> myriad-config-default.yml. Can you double check if all configuration are
> correct and the permissions are OK as well?
>
> -Sarjeet
>
> On Fri, Dec 4, 2015 at 10:53 AM, Prabhu Inbarajan <
> [email protected]> wrote:
>
> > I followed the myriad setup instructions , and was able to get resource
> > manager invoke the myriad scheduler and talk to the mesos master. But I
> > see the following error in the mesos slave logs and my yarn submissions
> are
> > stuck.
> >
> > My setup is as follows:
> > 1. Hadoop 2.7.1
> > 2. Jdk8
> > 3. Mesos Version: 0.25.0
> > 4. 1 master + 2 slaves
> > 5. ubuntu 14.04 + Kernel Linux master.dev 3.19.0-33-generic
> > #38~14.04.1-Ubuntu SMP Fri Nov 6 18:17:28 UTC 2015 x86_64 x86_64 x86_64
> > GNU/Linux
> >
> > Given this team is running with this, it is hard for me to presume this
> is
> > a argument overflow issue and would require somekind of a kernel
> recompile
> > : http://www.linuxjournal.com/article/6060?page=0,0. I am also thinking
> if
> > to recompile mesos for better diagnostics. the subprocess.cpp seems to
> have
> > better logging in master :
> >
> >
> https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/subprocess.cpp
> > than in 0.25.0
> >
> >
> >
> > ABORT:
> > (/tmp/mesos-build/mesos-repo/3rdparty/libprocess/src/subprocess.cpp:177):
> > Failed to os::execvpe in childMain: Argument list too long*** Aborted
> > at 1449220361 (unix time) try "date -d @1449220361" if you are using
> > GNU date ***
> > PC: @ 0x7fbfd2c66cc9 (unknown)
> > *** SIGABRT (@0x231d) received by PID 8989 (TID 0x7fbfc944a700) from
> > PID 8989; stack trace: ***
> > @ 0x7fbfd3005340 (unknown)
> > @ 0x7fbfd2c66cc9 (unknown)
> > @ 0x7fbfd2c6a0d8 (unknown)
> > @ 0x40a902 _Abort()
> > @ 0x40a93c _Abort()
> > @ 0x7fbfd477ac3b process::childMain()
> > @ 0x7fbfd477cc6d std::_Function_handler<>::_M_invoke()
> > @ 0x7fbfd2d2a47d (unknown)
> >
>
