Hi there, Apologies for the unusual post but in the absence of a user list I am sort of left without other options. :-)
I have been keeping an eye on Myriad as we run a number of Hadoop nodes next to a bunch of some Mesos friendly use-cases and currently struggle to manage resourcing boundaries. I have asked this question to a number of different individuals at MapR but never managed to get a clear answer, therefore apologies for the somehow MapR focused question but.... How would a MapR NM container connect back to Yarn without running MFS inside the container? Reason I ask is simple: By default MapR NM nodes depend on MFS in order to create a local volume. This applies also to compute nodes and is explicitly documented behaviour and seen on the logs on entries similar to the following: 2015-09-10 11:11:13,794 INFO com.mapr.hadoop.mapred.LocalVolumeAuxService: Checking for local volume. If volume is not present command will create and mount it. Failure to do so (using the default config) will usually (always?) result a failure of the NM to start. I imagine this behaviour may be worked around but I could not find any documentation about it. If that is not the case, this will have some interesting consequences on MapR users adopting Myriad. Taking for example users opting for following an MFS backed Mesos topology like the one described in here: https://www.mapr.com/blog/my-experience-running-docker-containers-on-mesos https://www.mapr.com/solutions/zeta-enterprise-architecture My understanding is that a good number of baremetal servers would be running MFS and ensuring that resources (disk controllers and disk slots in this case) are utilised to their maximum potential. So far so good. But when you take this with the com.mapr.hadoop.mapred.LocalVolumeAuxService behaviour mentioned above, one would end up having to run MFS on both the baremetal slave and within the container it is hosting! This raises a few interesting questions: * How to deal with MFS service and its memory gluttony? Caching would potentially be happening on both baremetal and containers... * Both barebone and contained would have to contact the CLDB in order to function well... Lets say this would likely cause MapR share prices would go up. :-) Keen to understand from the MapR commiters what is the view / work around createTTvolume.sh Kind regards Andre
