Please,can any one give me guidelines or help for improving hadoop performance
by improving the scheduling algorithm andwhich classes are responsible for the
scheduling
Hi Amr,
Hadoop contains the FIFO scheduler and two more advanced schedulers, the
capacity scheduler, developed at Yahoo, and the fair scheduler, developed
at Facebook.
In Hadoop 2, the fair scheduler code is located in
the org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair package,
and
Ivan Mitic created HADOOP-9490:
--
Summary: LocalFileSystem#reportChecksumFailure not closing the
checksum file handle before rename
Key: HADOOP-9490
URL: https://issues.apache.org/jira/browse/HADOOP-9490
Hi,
Addition to Sandy, there is one more scheduler called HOD (Hadoop on
Demand). Please go through the following links to get more details on
schedulers.
HOD - http://hadoop.apache.org/docs/r1.1.2/hod_scheduler.html
Fair - http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html
Capacity - http:
Would an application compiled against YARN/MRv2 run transparently on MRv1?
Are there any API differences ?
How stable is YARN/MRV2?
Best Regards
Amir Sanjar
System Management Architect
PowerLinux Open Source Hadoop development lead
IBM Senior Software Engineer
Phone# 512-286-8393
Fax# 512-83
As par my experience the MapReduce API is same for both YARN and MRv2.
Applications compiled again YARN should run smoothly on MRv1. Ans the vice
versa is also true.
And in general YARN is pretty stable now.
Regards,
Shekhar
On Sun, Apr 21, 2013 at 3:45 PM, Amir Sanjar wrote:
> Would an appli
thanks Shekhar, do you know when we will have an stable release for hadoop
2.0 (not alpha)?
Also from you experience, what component of hadoop 2.0.3 are most unstable
and are more likely need more attention?
Best Regards
Amir Sanjar
System Management Architect
PowerLinux Open Source Hadoop devel
I am sorry Amir. I don't have answers for these questions. Because I don't
use Hadoop for any real production jobs.
Mainly I play with the Scheduler and ResourceManager of YARN as part of my
thesis. So I just run some simple jobs to test the performance of the
scheduler.
On Sun, Apr 21, 2013 at 4
