Hi Mesos Community, I am a bit surprised to see that no one has done a framework to run HDFS on top of Mesos directly since a lot of people seem to use HDFS in the community. HDFS seems to be managed separately from Mesos (but will probably run on the same machines). Is there any reason for that?
My understanding is that using Mesos to manage all resources and having HDFS on top of it makes much more sense (just like a FS runs inside an OS, not on the side). Is it technical complexity? (we run HDFS and YARN with HA, journalnodes and Kerberos Security and it is definitely a beast) Is it because no one really feels the need for this since they are already running HDFS on the side close to the hardware and don't want to waste time having it in Mesos? Best Maxime
