We've a 20 node mesos cluster running mesos v0.21.1, We run marathon on top
of this setup without any problems for ~4 months now. I'm now trying
to get hadoop
mesos <https://github.com/mesos/hadoop/> integration working but I see the
TaskTrackers that gets launched are failing with the following error
I0818 05:36:35.058688 24428 fetcher.cpp:409] Fetcher Info:
{"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/20150706-075218-1611773194-5050-28439-S473\/hadoop","items":[{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"hdfs:\/\/hdfs.prod:54310\/user\/ashwanth\/hadoop-with-mesos-2.6.0-cdh5.4.4.tar.gz"}}],"sandbox_directory":"\/var\/lib\/mesos\/slaves\/20150706-075218-1611773194-5050-28439-S473\/frameworks\/20150706-075218-1611773194-5050-28439-4532\/executors\/executor_Task_Tracker_4129\/runs\/c26f52d4-4055-46fa-b999-11d73f2096dd","user":"hadoop"}
I0818 05:36:35.059806 24428 fetcher.cpp:364] Fetching URI
'hdfs://hdfs.prod:54310/user/ashwanth/hadoop-with-mesos-2.6.0-cdh5.4.4.tar.gz'
I0818 05:36:35.059821 24428 fetcher.cpp:238] Fetching directly into the
sandbox directory
I0818 05:36:35.059835 24428 fetcher.cpp:176] Fetching URI
'hdfs://hdfs.prod:54310/user/ashwanth/hadoop-with-mesos-2.6.0-cdh5.4.4.tar.gz'
*mesos-fetcher:
/tmp/mesos-build/mesos-repo/3rdparty/libprocess/3rdparty/stout/include/stout/try.hpp:90:
const string& Try<T>::error() const [with T = bool; std::string =
std::basic_string<char>]: Assertion `data.isNone()' failed.*
*** Aborted at 1439876195 (unix time) try "date -d @1439876195" if you are
using GNU date ***
PC: @ 0x343ee32635 (unknown)
*** SIGABRT (@0x5f6c) received by PID 24428 (TID 0x7f988832f820) from PID
24428; stack trace: ***
@ 0x343f20f710 (unknown)
@ 0x343ee32635 (unknown)
@ 0x343ee33e15 (unknown)
@ 0x343ee2b75e (unknown)
@ 0x343ee2b820 (unknown)
@ 0x408b0a Try<>::error()
@ 0x40cbcf download()
@ 0x4098a3 main
@ 0x343ee1ed5d (unknown)
@ 0x40aeb5 (unknown)
Failed to synchronize with slave (it's probably exited)
Environment
- EC2 Machines
- Output of lsb_release -a
LSB Version:
:base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description: CentOS release 6.5 (Final)
Release: 6.5
Codename: Final
Any ideas what I'm doing wrong?
--
-- Ashwanth Kumar
