[
https://issues.apache.org/jira/browse/YARN-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14027224#comment-14027224
]
Tsuyoshi OZAWA commented on YARN-1514:
--------------------------------------
Updated a patch. This patch includes following changes:
* Adding dependency on zookeeper-tests' jar.
* Made number of applications, number of application attempts , and ZK
connection configurable.
* Added YarnTestDriver to launch this perf tool from bin/hadoop command.
Example of running jar is as follows:
{code}
bin/hadoop jar
../hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/hadoop-yarn-server-resourcemanager-3.0.0-SNAPSHOT-tests.jar
TestZKRMStateStorePerf -appSize 100 -appattemptsize 100 -hostPort
localhost:2181
{code}
One problem is too much logging. If we run the command, the following logs are
generated:
{quote}
...
14/06/10 23:25:55 INFO recovery.RMStateStore: Storing info for app:
application_1352994193343_0099
14/06/10 23:26:31 INFO recovery.TestZKRMStateStore: ZKRMStateStore takes 2791
msec to loadState.
....
ZKRMStateStore takes 2791 msec to loadState.
...
14/06/10 23:26:31 INFO recovery.ZKRMStateStore: Watcher event type: NodeDeleted
with state:SyncConnected for
path:/Test/ZKRMStateRoot/RMAppRoot/application_1352994193343_0000 for Service
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore in state
org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: STARTED
{quote}
This is because of ZKRMStateStore generates log via {{LOG.info()}}.
[~kkambatl], what do you think about this? Should we improve it on this JIRA?
> Utility to benchmark ZKRMStateStore#loadState for ResourceManager-HA
> --------------------------------------------------------------------
>
> Key: YARN-1514
> URL: https://issues.apache.org/jira/browse/YARN-1514
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Tsuyoshi OZAWA
> Assignee: Tsuyoshi OZAWA
> Fix For: 2.5.0
>
> Attachments: YARN-1514.1.patch, YARN-1514.wip-2.patch,
> YARN-1514.wip.patch
>
>
> ZKRMStateStore is very sensitive to ZNode-related operations as discussed in
> YARN-1307, YARN-1378 and so on. Especially, ZKRMStateStore#loadState is
> called when RM-HA cluster does failover. Therefore, its execution time
> impacts failover time of RM-HA.
> We need utility to benchmark time execution time of ZKRMStateStore#loadStore
> as development tool.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
