Have you taken a look at https://issues.apache.org/jira/browse/YARN-326 ?
-- Hitesh On Sep 6, 2013, at 11:02 AM, hilfi alkaff wrote: > Thanks for all the replies. I think I have found the relevant codes that I > would like to modify. That said, a project that I'm doing now requires > containers to have network bandwidth as one of its resources (In > Resource.java: it currently only models memory). > > Since I'm planning to implement it anyway, I hope to be able to help > Hadoop's development. However, I could not find the relevant JIRA for this. > If you know of an existing ticket that is relevant to the aforementioned > issue, let me know. If there is none, should I make my changes first (as > listed http://wiki.apache.org/hadoop/HowToContribute) and get back after > I'm done with my code? > > Thanks in advance. > > > On Fri, Sep 6, 2013 at 6:37 AM, Steve Loughran <ste...@hortonworks.com>wrote: > >> worth adding is that this can generate a bias towards affinitive >> assignment of an apps containers; for the YARN-896 service we've put >> anti-affinity as a subtask, along with having AM opt to get notifications >> if assignments can't be met in a bounded period (or it could just examine >> its queue of outstanding requests and reach the same conclusion based on >> when the requests were submitted) >> >> >> On 6 September 2013 07:43, Sandy Ryza <sandy.r...@cloudera.com> wrote: >> >>> That's right. Nodes keep checking in and, when they do, the >>> ResourceManager looks for outstanding requests. This means that >> assignment >>> of containers to nodes depends on the order that they heartbeat in. If >>> container requests come in for specific nodes locality is achieved >> through >>> delay scheduling - the ResourceManager will wait for a configurable >> number >>> of heartbeats before assigning a container to a non-local node. If >> strict >>> locality is turned on, the ResourceManager will wait indefinitely for a >>> local node. >>> >>> -Sandy >>> >>> >>> On Fri, Sep 6, 2013 at 3:33 PM, hilfi alkaff <hilfialk...@gmail.com> >>> wrote: >>> >>>> I see. What I'm wondering about is; when an application master tries to >>>> request a container from resource manager, which part of the code in >> the >>>> resource manager actually decide which node to fetch this container >> from. >>>> Is this step being done asynchronously (ie: Nodes keep checking if >> there >>>> are requests from the ResourceManager during the node update event?) >>>> >>>> >>>> On Fri, Sep 6, 2013 at 1:22 AM, Sandy Ryza <sandy.r...@cloudera.com> >>>> wrote: >>>> >>>>> Hi Hilfi, >>>>> >>>>> Nodes are constantly heartbeating to the ResourceManager. A node >>> update >>>>> event is triggered each time this happens. >>>>> >>>>> -Sandy >>>>> >>>>> >>>>> On Fri, Sep 6, 2013 at 3:20 PM, hilfi alkaff <hilfialk...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm trying to trace the code flow on the scheduling done in YARN. I >>>> would >>>>>> like to know where the code that does which node to schedule for >> the >>>>> jobs. >>>>>> >>>>>> I found the handle() function in the resource manager's scheduler >>> (eg: >>>>>> CapacityScheduler.java) that handles node update event which then >>>>> executes >>>>>> the assignment of containers for that particular node, but I do not >>>>>> understand how that node even get chosen. >>>>>> >>>>>> If anybody could tell me about a file, function or module name that >>>> does >>>>>> this, that would be extremely helpful. >>>>>> >>>>>> -- >>>>>> ~Hilfi Alkaff~ >>>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> ~Hilfi Alkaff~ >>>> >>> >> >> -- >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity to >> which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> > > > > -- > ~Hilfi Alkaff~
