On Oct 20, 2014, at 2:09 AM, Ankur Chauhan <[email protected]> wrote:
> Hi all, > > I am trying to setup a new mesos cluster and I so far I have a set of master > and slave nodes working and I can get everything running. I am able to > install and run a couple of sample apps, hookup jenkins etc. My main question > now is Do I really need HDFS? All my artifacts (for apps) are on a protected > S3 bucket or in a private docker registry. > > If I need HDFS, do I need to go "all in" even when I am not using hdfs as a > data store but rather as a simple way to fetch files from s3; or can I get > away with putting the correct core-site.xml and hdfs-site.xml in HADOOP_HOME > and get away with it? We are running Mesos entirely without HDFS with no problems. We use Docker to distribute our application to slave nodes, and keep no state on individual nodes.
