On Sun, Oct 10, 2010 at 7:22 PM, Ken Goodhope <[email protected]> wrote: > Another option might be to extend fair scheduler's task selector. The > code is already set up to make this a configurable abstraction point. > One word of caution, if you are using the cloudera distro, you will > find that fair scheduler has been modified to not use those classes > anymore. Something that wasn't documented very well. Still works with > 20.2. Not sure about 21.
Just to clarify, we don't have any custom non-Apache code in our fair scheduler in CDH. Rather, we've backported newer fair scheduler work from 0.21 and trunk into 0.20. Most importantly, we backported MAPREDUCE-706 which changed a lot of the internals. Thanks -Todd > > On Thursday, October 7, 2010, Jeff Zhang <[email protected]> wrote: >> Currently, TaskTracker send heatbeat to JobTracker, and JobTracker >> will send actions to TaskTracker according TaskTracker's status. I >> think you should do some hacking about the heatbeat part of JobTracker >> >> public synchronized HeartbeatResponse heartbeat(TaskTrackerStatus status, >> boolean restarted, >> boolean initialContact, >> boolean acceptNewTasks, >> short responseId) >> >> >> >> On Fri, Oct 8, 2010 at 10:38 AM, Shen LI <[email protected]> wrote: >>> Hi, Thanks you very much for your reply. I want to run my own algorithm for >>> this part to see if we can achieve better outcome in specific scenario. So >>> how can I modify it? >>> Thanks a lot! >>> Shen >>> >>> On Thu, Oct 7, 2010 at 6:33 PM, Jeff Zhang <[email protected]> wrote: >>>> >>>> I believe it is possible. But what is your purpose? I believe current >>>> solution is good enough. >>>> >>>> >>>> >>>> >>>> >>>> On Fri, Oct 8, 2010 at 2:57 AM, Shen LI <[email protected]> wrote: >>>> > Hi, >>>> > How can I modify the task assignment strategy in hadoop which is used to >>>> > assign tasks to different worker nodes? (Not the job scheduler) >>>> > Big thanks, >>>> > Shen >>>> >>>> >>>> >>>> -- >>>> Best Regards >>>> >>>> Jeff Zhang >>> >>> >> >> >> >> -- >> Best Regards >> >> Jeff Zhang >> > -- Todd Lipcon Software Engineer, Cloudera
