[
https://issues.apache.org/jira/browse/SAMZA-58?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13961290#comment-13961290
]
Chris Riccomini commented on SAMZA-58:
--------------------------------------
So, I think the things that need to be updated are:
# Make DEFAULT_POLL_INTERVAL_MS configurable. You can add a
yarn.am.poll.interval.ms config to YarnConfig.scala.
# The while-loop in run() seems strange now, since we're not heart-beating it
anymore. Should we re-think this section of code?
For (2), I think it's safe for us to remove the onEventLoop callback in the
YarnAppMasterListener interface. If we do this, it seems to me that we don't
really need that while loop anymore. I'm not sure how the other new async AMs
handle this, but we essentially just want to amClient.start, listen for some
form of a shutdown event (some exception, or shouldShutdown==true), and then
call amClient.stop. I wish the amClient itself had a heartbeat callback. Do
most other async AMs just sit in a loop like this as well?
> Use YARN's AMRMClientAsync client library
> -----------------------------------------
>
> Key: SAMZA-58
> URL: https://issues.apache.org/jira/browse/SAMZA-58
> Project: Samza
> Issue Type: Bug
> Components: yarn
> Affects Versions: 0.6.0
> Reporter: Chris Riccomini
> Assignee: Zhijie Shen
> Attachments: SAMZA-58.1.patch
>
>
> YARN 2.2.0 has a nice Async API for clients and AMs. This API didn't exist
> when we did the initial YARN integration for Samza. We should upgrade Samza
> to use these new APIs.
> The API is loosely based off Samza's own AM code, so we can probably strip
> out a lot of it (YarnAppMaster, mainly), and switch everything over to the
> call-back based API.
> For details, see:
> https://issues.apache.org/jira/browse/YARN-417
> This new API is used in DistributedShell now, so we can use that for testing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
