[
https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587233#comment-13587233
]
Chris Riccomini commented on YARN-417:
--------------------------------------
> My first comment would be to see if we can provide this functionality via a
> single object instead of a combination of AMRMClient, Heartbeater and
> AllocationListener. Every new object and its combination with others
> increases mental complexity.
Yeah, this was my initial inclination as well. I wanted some way to eliminate
the AMRMClient (applicationMasterHelper, in my case) by having the callbacks
return objects that signaled what they wanted. I couldn't come up with a scheme
that was simpler than just using the AMRMClient, though. If you guys can come
up with something that works, and is clean, I'm all ears.
> Also, while callbacks are a reasonable approach, it would be good to spend a
> little time thinking about other notification mechanisms like events or
> futures.
Hmm, this is an interesting thought. Generally, I find futures easier to work
with, but as Sandy pointed out, I'm not sure how well it fits this model, where
you get updates that trickle in. It almost seems like a CountDownLatch for a
resource request could be used, but that seems more complex to deal with than a
callback. To riff on Sandy's API example, I could imagine something like:
{code}
List<Container> myAllocatedContainers =
heartbeater.submitContainerRequests().getContainers(3);
{code}
To wait for three containers before proceeding. You could also have
getContainers(), which would wait for all. This is very CountDownLatch-ish. I'm
not entirely sure how you might model the onReboot and failure/complete
scenarios, though. Thoughts? My imagination is failing me.
> Something else to think about. completedContainers and allocatedContainers
> are essentially coming at the same time. Breaking them into 2 calls may open
> up ordering issues.
I had the same inclination that Sandy did, which was that I can see the need
for onComplete* before onAllocated, so that I can properly re-assign work to
allocated containers if an existing container failed, but beyond that, I
couldn't come up with a reason why you might want onAllocated before
onComplete*. I guess the question isn't so much about ordering, it's really: is
there ever a case where you need to know both what was allocated AND what was
completed at the same time, or in total (i.e. not one at a time)? I can't think
of anything.
> Add a poller that allows the AM to receive notifications when it is assigned
> containers
> ---------------------------------------------------------------------------------------
>
> Key: YARN-417
> URL: https://issues.apache.org/jira/browse/YARN-417
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, applications
> Affects Versions: 2.0.3-alpha
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: YARN-417.patch, YarnAppMaster.java,
> YarnAppMasterListener.java
>
>
> Writing AMs would be easier for some if they did not have to handle
> heartbeating to the RM on their own.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
