Building custom block placement policy. What is srcPath?

2014-07-24 Thread Arjun Bakshi
Hi, I want to write a block placement policy that takes the size of the file being placed into account. Something like what is done in CoHadoop or BEEMR paper. I have the following questions: 1- What is srcPath in chooseTarget? Is it the path to the original un-chunked file, or it is a path

Re: Building custom block placement policy. What is srcPath?

2014-07-24 Thread Harsh J
Hello, (Inline) On Thu, Jul 24, 2014 at 11:11 PM, Arjun Bakshi baksh...@mail.uc.edu wrote: Hi, I want to write a block placement policy that takes the size of the file being placed into account. Something like what is done in CoHadoop or BEEMR paper. I have the following questions: 1-

Re: Building custom block placement policy. What is srcPath?

2014-07-24 Thread Arjun Bakshi
Hi, Thanks for the reply. It cleared up a few things. I hadn't thought of situations of under-replication, but I'll give it some thought now. It should be easier since, as you've mentioned, by that time the namenode knows all the blocks that came from the same file as the under-replicated

Re: Building custom block placement policy. What is srcPath?

2014-07-24 Thread Harsh J
Hi, Inline. On Fri, Jul 25, 2014 at 2:55 AM, Arjun Bakshi baksh...@mail.uc.edu wrote: Hi, Thanks for the reply. It cleared up a few things. I hadn't thought of situations of under-replication, but I'll give it some thought now. It should be easier since, as you've mentioned, by that time