Re: how to add a shareable node label?

2016-10-06 Thread Sunil Govind
Hi Frank I think as of today this is not possible. You could try and experience the "non-exlusive" feature of node-label which will officially come in 2.8 soon. Or you can try it in "Hadoop 3 alpha1" release too if its fine to check. YARN-3214 has

Block Replace Operation in Mover/Balancer

2016-10-06 Thread Senthil Kumar
Hi All , Recently we have upgraded our Cluster to Hadoop 2.7.1 version. When i try to run Mover , it throws below exception : 2016-10-06 00:24:44,643 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: opCopyBlock BP-1267816582-10.114.118.11-1392844883031:blk_12046985669_160995799

S3AFileSystem & read-after-write consistency

2016-10-06 Thread Dave Maughan
Hi, I'm investigating S3's read-after-write consistency model with S3AFileSystem and something is not quite clear to me, so I'm hoping someone more knowledgeable can clarify it for me. Amazon state ( http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html): *"Amazon S3 provides

Newbie Ambari Question

2016-10-06 Thread Deepak Goel
Hey Namaskara~Nalama~Guten Tag~Bonjour Sorry, Is this the right forum for asking a question "Ambari Hadoop Installation" from Hortonworks? Thanks Deepak -- Keigu Deepak 73500 12833 www.simtree.net, dee...@simtree.net deic...@gmail.com LinkedIn: www.linkedin.com/in/deicool Skype:

Hadoop: precomputing data

2016-10-06 Thread venito camelas
I'm designing a prototype using *Hadoop* for video processing to do face recognition. I thought of 2 ways of doing it. *Approach 1:* I was thinking of doing something in 2 steps: 1. A map that receives frames and if a face is found it gets stored for the next step. 2. A map that

RE: how to add a shareable node label?

2016-10-06 Thread Frank Luo
Sunil, You confirmed my understanding. I got the understanding by reading the docs and haven’t really tried 2.8 or 3.0-alphal. My situation is that I am in a multi-tenant env, and got several very powerful machines with expensive licenses to run a particular linux job, let’s say Job_A. But

RE: how to add a shareable node label?

2016-10-06 Thread Frank Luo
Thanks Sunil. Ø 3. If there is any future ask for those resources , we will preempt the non labeled apps and give them back to labeled apps. Unfortunately, I am still not able to use it, because of the preemptive behavior. The jobs that steals labelled resources are not preemptable, and I’d