If one is building a mesos-cluster, then is a DFS mandatory, or is there combinations of other codes that suffice for the needs of a mesos cluster?
So what is the list of available Distributed File Systems that are generally available and mesos is know to work on top of?
Glusterfs, Lustrefs, FhGFS (BeeGFS), Ceph [1] http://en.wikipedia.org/wiki/Comparison_of_distributed_file_systems An explicit list of other DFS's or combinations of codes that provide these required 'feature sets' is of interest to many. James On 10/20/14 07:08, David Greenberg wrote:
You certainly don't need hdfs if you've got other infrastructure that's providing the same sorts of features. We don't run hdfs on our mesos cluster, and it's fine. On Monday, October 20, 2014, Ankur Chauhan <[email protected] <mailto:[email protected]>> wrote: __ Hi all, I am trying to setup a new mesos cluster and I so far I have a set of master and slave nodes working and I can get everything running. I am able to install and run a couple of sample apps, hookup jenkins etc. My main question now is Do I really need HDFS? All my artifacts (for apps) are on a protected S3 bucket or in a private docker registry. If I need HDFS, do I need to go "all in" even when I am not using hdfs as a data store but rather as a simple way to fetch files from s3; or can I get away with putting the correct core-site.xml and hdfs-site.xml in HADOOP_HOME and get away with it? It would really help how other have their mesos setup in production or what they would recommend regarding my setup? -- Ankur Chauhan
