[
https://issues.apache.org/jira/browse/YARN-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597898#comment-16597898
]
Eric Yang commented on YARN-8733:
---------------------------------
When dependency check happens at component level, there is some chance of risk
that component might be running, but the service as a whole is not running.
Incorrect use of remote component check can result in false positive response.
Yet, the benefit of doing remote dependency check at component level is to
boost parallelization efficiency. A more concrete example is Spark depends on
the same remote database that hive uses for Spark SQL. Spark service only
needs to wait until the database is ready without waiting for the rest of the
hive containers to become ready. On the other hand, HBase master depends on
HDFS service being ready, instead of namenode port being ready. HBase service
will have to wait until HDFS service is ready to proceed with region servers
deployment. We will address service dependency in YARN-8734. Most likely that
component dependency can be built at component level for efficiency reason, but
built at service level is safer to prevent race conditions that user might not
have known in advance.
> Readiness check for remote component
> ------------------------------------
>
> Key: YARN-8733
> URL: https://issues.apache.org/jira/browse/YARN-8733
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn-native-services
> Reporter: Eric Yang
> Priority: Major
>
> When a service is deploying, there can be remote component dependency between
> services. For example, Hive server 2 can depend on Hive metastore, which
> depends on a remote MySQL database. It would be great to have ability to
> check the remote server and port to make sure MySQL is available before
> deploying Hive LLAP service.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
