Saurabh, > i am experimenting with hadoop. wanted to ask that is the Task distribution > policy by job tracker pluggable if yes where in the code tree is it defined. >
Take a look at o.a.h.mapred.TaskScheduler. That's the abstract class that needs to be extended to define a new scheduling policy. Also, please do take a look at the existing schedulers that extend this class. There are 3-4 implementations including the default scheduler, capacity scheduler, fairshare scheduler and dynamic priority scheduler. It may be worthwhile to see if your ideas match any of the existing implementations to some degree and then consider enhancing those as a first option. Thanks Hemanth
