And for your this question: >on instances, that also contain hdfs service, to prevent all Data going over the network ? If you open HDFS Short-Circuit Local Reads, HDFS would auto read from local machine instead of read from network when the data exists in local machine.
On Tue, Jun 23, 2015 at 8:58 PM, haosdent <[email protected]> wrote: > For your second question, I think unless you already have data replica in > new hdfs datanode, otherwise hdfs would copy the block from other exist > datanode. > > On Tue, Jun 23, 2015 at 7:51 PM, Sebastien Brennion < > [email protected]> wrote: > >> Hi, >> >> >> - I would like to know it there is a way to make mesos dispatch in >> priority spark jobs, on instances, that also contain hdfs service, to >> prevent all Data going over the network ? >> >> >> - What I also not sure about, is how to handle the storage, if hdfs is >> running in mesos ? What happend when hdfs_1 is moved from mesos_worker_1 to >> mesos_worker_2, do all Data have to be copied ? How are handling this ? >> >> >> Regards >> >> Sébastien >> > > > > -- > Best Regards, > Haosdent Huang > -- Best Regards, Haosdent Huang

