[
https://issues.apache.org/jira/browse/HBASE-16375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15446175#comment-15446175
]
Loknath Priyatham Teja Singamsetty edited comment on HBASE-16375 at 8/29/16
3:33 PM:
--------------------------------------------------------------------------------------
The actual test resides in phoenix which utilises the added configuration for
e2e test of async secondary index which uses RM resource manager web address.
I am not sure if we need any tests to be added for this in HBase. If needed,
please let me know which test to be replicated in hbase test suite.
was (Author: singamteja):
The actual test resides in phoenix which utilises the added configuration for
e2e test of async secondary index which uses RM resource manager web address.
I am not sure if we need any tests to be added for this. If needed, please let
me know which test to be replicated in hbase test suite.
> Mapreduce mini cluster using HBaseTestingUtility not setting correct
> resourcemanager and jobhistory webapp address of MapReduceTestingShim
> --------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-16375
> URL: https://issues.apache.org/jira/browse/HBASE-16375
> Project: HBase
> Issue Type: Bug
> Affects Versions: 1.2.0
> Reporter: Loknath Priyatham Teja Singamsetty
> Assignee: Loknath Priyatham Teja Singamsetty
> Priority: Minor
> Fix For: 2.0.0, 1.2.0, 1.3.0, 1.4.0, 1.2.2, 1.1.6, 1.3.1, 1.2.3,
> 0.98.22, 1.2.4
>
> Attachments: HBASE-16375_0.98_and_above.patch
>
>
> Starting mapreduce mini cluster using HBaseTestingUtility is not setting
> "yarn.resourcemanager.webapp.address" and
> "mapreduce.jobhistory.webapp.address" which are required for getting the
> submitted yarn apps using mapreduce webapp. These properties are not being
> copied from jobConf of MapReduceTestingShim resulting in default values.
> {quote}
> HBaseTestingUtility.java
> // Allow the user to override FS URI for this map-reduce cluster to use.
> mrCluster = new MiniMRCluster(servers,
> FS_URI != null ? FS_URI : FileSystem.get(conf).getUri().toString(), 1,
> null, null, new JobConf(this.conf));
> JobConf jobConf = MapreduceTestingShim.getJobConf(mrCluster);
> if (jobConf == null) {
> jobConf = mrCluster.createJobConf();
> }
> jobConf.set("mapreduce.cluster.local.dir",
> conf.get("mapreduce.cluster.local.dir")); //Hadoop MiniMR overwrites
> this while it should not
> LOG.info("Mini mapreduce cluster started");
> // In hadoop2, YARN/MR2 starts a mini cluster with its own conf instance
> and updates settings.
> // Our HBase MR jobs need several of these settings in order to properly
> run. So we copy the
> // necessary config properties here. YARN-129 required adding a few
> properties.
> conf.set("mapreduce.jobtracker.address",
> jobConf.get("mapreduce.jobtracker.address"));
> // this for mrv2 support; mr1 ignores this
> conf.set("mapreduce.framework.name", "yarn");
> conf.setBoolean("yarn.is.minicluster", true);
> String rmAddress = jobConf.get("yarn.resourcemanager.address");
> if (rmAddress != null) {
> conf.set("yarn.resourcemanager.address", rmAddress);
> }
> String historyAddress = jobConf.get("mapreduce.jobhistory.address");
> if (historyAddress != null) {
> conf.set("mapreduce.jobhistory.address", historyAddress);
> }
> String schedulerAddress =
> jobConf.get("yarn.resourcemanager.scheduler.address");
> if (schedulerAddress != null) {
> conf.set("yarn.resourcemanager.scheduler.address", schedulerAddress);
> }
> {quote}
> As a immediate fix for phoenix e2e test to succeed, need the below lines to
> be added as well
> {quote}
> String rmWebappAddress =
> jobConf.get("yarn.resourcemanager.webapp.address");
> if (rmWebappAddress != null) {
> conf.set("yarn.resourcemanager.webapp.address", rmWebappAddress);
> }
> String historyWebappAddress =
> jobConf.get("mapreduce.jobhistory.webapp.address");
> if (historyWebappAddress != null) {
> conf.set("mapreduce.jobhistory.webapp.address", historyWebappAddress);
> }
> {quote}
> Eventually, we should also see if we can copy over all the jobConf properties
> to HBaseTestingUtility conf object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
