Vinod, For sure they can connect with each other. I can see the slave on the webUI for about a minute or so then it just dropped out of cluster. Enclosed are the slave output and master output in that one minute. Maybe you can spot something that I am not doing right.
Thanks, Scott On Mon, Oct 1, 2012 at 7:49 PM, Vinod Kone <[email protected]> wrote: > Looks like the slave is not responding to health checks from the master. Is > the network connection from master-->slave alright? is the machine hosting > slave is cpu starved? Those are some of the things, I would check for. > > @vinodkone > > > On Mon, Oct 1, 2012 at 6:04 PM, Scott Wang < > [email protected]> wrote: > >> I am trying to setup a small cluster a master and a slave but I am >> getting the following output and the slave just terminated. >> >> ------------------------------------------------Slave >> >> output----------------------------------------------------------------------- >> I1002 01:00:02.868795 27122 slave.cpp:1160] Current disk usage 2.09%. >> Max allowed age: 6.85days >> I1002 01:00:17.974647 27123 slave.cpp:335] Slave asked to shut down >> I1002 01:00:17.974792 27123 slave.cpp:313] Slave terminating >> >> >> -----------------------------------------------Master >> >> output---------------------------------------------------------------------- >> W1002 01:00:17.967651 11433 master.cpp:1173] Removing slave >> 201210020057-1994437898-5050-11419-1 at itvm638:34013 because it has >> been deactivated >> I1002 01:00:17.968174 11433 master.cpp:1182] Master now considering a >> slave at itvm638:34013 as inactive >> I1002 01:00:17.968328 11435 hierarchical_allocator_process.hpp:371] >> Removed slave 201210020057-1994437898-5050-11419-1 >> >> Does anyone have any idea what I should do to prevent the slave going >> down by itself. >> >> Thanks, >> Scott >>
root@ItVm635:~/downloads/mesos/build/bin# ./mesos-master.sh --cluster=rumble_test I1002 03:18:54.060497 18340 main.cpp:115] Build: 2012-10-01 19:23:53 by root I1002 03:18:54.060981 18340 main.cpp:116] Starting Mesos master I1002 03:18:54.061601 18354 master.cpp:303] Master started on 10.181.224.118:5050 I1002 03:18:54.061782 18354 master.cpp:318] Master ID: 201210020318-1994437898-5050-18340 W1002 03:18:54.062505 18357 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:18:54.065750 18354 master.cpp:560] Elected as master! I1002 03:18:54.097101 18359 webui.cpp:61] Loading webui script at '/root/downloads/mesos/build/../src/webui/master/webui.py' Bottle server starting up (using WSGIRefServer())... Listening on http://0.0.0.0:8080/ Use Ctrl-C to quit. I1002 03:18:55.063876 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 32.16us I1002 03:18:56.065438 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 3.95us I1002 03:18:57.066777 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 4.09us I1002 03:18:58.068367 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 3.29us W1002 03:18:59.063473 18354 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:18:59.069622 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 2.79us I1002 03:19:00.070927 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 2.83us I1002 03:19:01.072056 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 3.29us I1002 03:19:02.073478 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 2.98us I1002 03:19:02.930248 18354 master.cpp:924] Attempting to register slave on itvm638.cdrentertainment.com at slave(1)@65.61.189.78:45730 I1002 03:19:02.930362 18354 master.cpp:1160] Master now considering a slave at itvm638.cdrentertainment.com:45730 as active I1002 03:19:02.930416 18354 master.cpp:1740] Adding slave 201210020318-1994437898-5050-18340-0 at itvm638.cdrentertainment.com with cpus=2; mem=2979; disk=144580 I1002 03:19:02.931062 18354 hierarchical_allocator_process.hpp:345] Added slave 201210020318-1994437898-5050-18340-0 (itvm638.cdrentertainment.com) with cpus=2; mem=2979; disk=144580 (and cpus=2; mem=2979; disk=144580 available) I1002 03:19:02.931154 18354 hierarchical_allocator_process.hpp:561] Performed allocation for slave 201210020318-1994437898-5050-18340-0 in 1.81us I1002 03:19:03.074321 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 8.25us W1002 03:19:04.064885 18357 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:04.075013 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 8.87us I1002 03:19:05.076375 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.47us I1002 03:19:06.078531 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 9.72us I1002 03:19:07.079926 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.36us I1002 03:19:08.081398 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 13.08us W1002 03:19:09.066542 18357 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:09.081683 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.45us I1002 03:19:10.083546 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.91us I1002 03:19:11.085108 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.88us I1002 03:19:12.086959 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.59us I1002 03:19:13.088510 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.02us W1002 03:19:14.067903 18355 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:14.089004 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 16.75us I1002 03:19:15.090610 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 15.72us I1002 03:19:16.092589 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.26us I1002 03:19:17.094243 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 16.36us I1002 03:19:18.095765 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 8.78us W1002 03:19:19.069990 18355 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:19.096144 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.98us I1002 03:19:20.097651 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.87us I1002 03:19:21.099164 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 11.30us I1002 03:19:22.100342 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.47us I1002 03:19:23.101855 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 11.38us W1002 03:19:24.071496 18354 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:24.102639 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 15.57us I1002 03:19:25.104126 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.54us I1002 03:19:26.105618 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 8.16us I1002 03:19:27.107084 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.64us I1002 03:19:28.107723 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 11.37us W1002 03:19:29.073935 18355 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:29.109031 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 13.99us I1002 03:19:30.109555 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 8.35us I1002 03:19:31.110970 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.46us I1002 03:19:32.112473 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.51us I1002 03:19:33.112993 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 15.78us W1002 03:19:34.076211 18355 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:34.114274 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 11.39us I1002 03:19:35.116025 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.66us I1002 03:19:36.117476 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.80us I1002 03:19:37.119616 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.62us I1002 03:19:38.121234 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 15.01us W1002 03:19:39.078431 18356 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:39.122545 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 15.28us I1002 03:19:40.123951 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.75us I1002 03:19:41.125656 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.01us I1002 03:19:42.127249 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 34.58us I1002 03:19:43.128568 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 13.81us W1002 03:19:44.080761 18355 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:44.129935 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 13.29us I1002 03:19:45.131304 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 59.24us I1002 03:19:46.133034 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.25us I1002 03:19:47.134496 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.94us I1002 03:19:48.135165 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.80us W1002 03:19:49.081715 18355 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:49.135871 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.64us I1002 03:19:50.137286 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.56us I1002 03:19:51.138329 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 9.98us I1002 03:19:52.139801 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.52us I1002 03:19:53.140770 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.93us W1002 03:19:54.083910 18356 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:54.142058 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 11.20us I1002 03:19:55.143578 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 7.98us I1002 03:19:56.145068 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 15.23us I1002 03:19:57.146302 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 21.42us I1002 03:19:58.147948 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 15.90us W1002 03:19:59.084988 18357 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:19:59.149277 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 15.04us I1002 03:20:00.150748 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.08us I1002 03:20:01.151746 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 10.76us I1002 03:20:02.153318 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 16.07us I1002 03:20:03.154062 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.29us W1002 03:20:04.086644 18357 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:20:04.154844 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 17.45us I1002 03:20:05.156390 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 12.56us I1002 03:20:06.158144 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 19.74us I1002 03:20:07.159842 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 13.70us I1002 03:20:08.161371 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 18.52us W1002 03:20:09.088462 18355 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:20:09.162686 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 19.06us I1002 03:20:10.165005 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 14.82us I1002 03:20:11.166882 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 18.85us I1002 03:20:12.168258 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 19.30us I1002 03:20:13.170023 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 19.63us W1002 03:20:14.089742 18356 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:20:14.171108 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 18.57us I1002 03:20:15.172893 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 16.11us I1002 03:20:16.175026 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 21.29us I1002 03:20:17.176830 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 1 slaves in 17.96us W1002 03:20:17.938084 18354 master.cpp:1173] Removing slave 201210020318-1994437898-5050-18340-0 at itvm638.cdrentertainment.com:45730 because it has been deactivated I1002 03:20:17.938537 18354 master.cpp:1182] Master now considering a slave at itvm638.cdrentertainment.com:45730 as inactive I1002 03:20:17.938683 18357 hierarchical_allocator_process.hpp:371] Removed slave 201210020318-1994437898-5050-18340-0 I1002 03:20:18.178428 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 4.67us W1002 03:20:19.091550 18356 master.cpp:78] No whitelist given. Advertising offers for all slaves I1002 03:20:19.179682 18354 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 4.36us I1002 03:20:20.180920 18357 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 3.69us I1002 03:20:21.182502 18356 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 4.59us I1002 03:20:22.184311 18355 hierarchical_allocator_process.hpp:543] Performed allocation for 0 slaves in 5.20us
root@itvm638:~/downloads/mesos/build/bin# ./mesos-slave.sh --master=itvm635.cdrentertainment.com:5050 --cluster=rumble_test I1002 03:19:02.876116 503 main.cpp:123] Creating "process" isolation module I1002 03:19:02.877226 503 main.cpp:131] Build: 2012-10-02 00:38:29 by root I1002 03:19:02.877313 503 main.cpp:132] Starting Mesos slave I1002 03:19:02.877954 517 slave.cpp:162] Slave started on 1)@65.61.189.78:45730 I1002 03:19:02.878092 517 slave.cpp:163] Slave resources: cpus=2; mem=2979; disk=144580 I1002 03:19:02.885445 520 slave.cpp:353] New master detected at [email protected]:5050 I1002 03:19:02.917531 522 webui.cpp:61] Loading webui script at '/root/downloads/mesos/build/../src/webui/slave/webui.py' I1002 03:19:02.970252 517 slave.cpp:373] Registered with master; given slave ID 201210020318-1994437898-5050-18340-0 I1002 03:19:02.971412 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-53742-26216-2 for removal I1002 03:19:02.971643 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-53742-26216-1 for removal I1002 03:19:02.971854 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-45965-26109-0 for removal I1002 03:19:02.972056 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-38056-26388-0 for removal I1002 03:19:02.972208 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-45965-26109-2 for removal I1002 03:19:02.972354 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-56087-25994-0 for removal I1002 03:19:02.972532 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-38056-26388-2 for removal I1002 03:19:02.972682 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020054-1321024833-51060-26576-2 for removal I1002 03:19:02.972930 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-38056-26388-1 for removal I1002 03:19:02.973136 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-53742-26216-0 for removal I1002 03:19:02.973345 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020054-1321024833-51060-26576-0 for removal I1002 03:19:02.973559 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020054-1321024833-56087-25994-0 for removal I1002 03:19:02.973767 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020054-1321024833-51060-26576-1 for removal I1002 03:19:02.973981 517 gc.cpp:97] Scheduling /tmp/mesos/slaves/201210020053-1321024833-45965-26109-1 for removal Bottle server starting up (using WSGIRefServer())... Listening on http://0.0.0.0:8081/ Use Ctrl-C to quit. I1002 03:20:02.881160 518 slave.cpp:1160] Current disk usage 2.09%. Max allowed age: 6.85days I1002 03:20:17.950041 519 slave.cpp:335] Slave asked to shut down I1002 03:20:17.950203 519 slave.cpp:313] Slave terminating
