Re: Does Samza work with ResourceManager in HA?

[Rick Mangi]

Hi John,

We just got this set up last week. I haven’t fully tested the failover but it 
certainly works for testing out our samza jobs on a pretty large cluster.

All of the server names are fqdn in our live yarn-site.xml

Hope this helps,

Rick


<property>
  <name>yarn.resourcemanager.ha.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.resourcemanager.cluster-id</name>
  <value>ops02</value>
</property>
<property>
  <name>yarn.resourcemanager.ha.rm-ids</name>
  <value>rm1,rm2</value>
</property>
<property>
  <name>yarn.resourcemanager.hostname.rm1</name>
  <value>yarnmaster01</value>
</property>
<property>
  <name>yarn.resourcemanager.hostname.rm2</name>
  <value>yarnmaster02</value>
</property>
<property>
  <name>yarn.resourcemanager.webapp.address.rm1</name>
  <value>yarnmaster01:8088</value>
</property>
<property>
  <name>yarn.resourcemanager.webapp.address.rm2</name>
  <value>yarnmaster02:8088</value>
</property>
<property>
  <name>yarn.resourcemanager.zk-address</name>
  <value>zk01:2181,zk02:2181,zk03:2181,zk04:2181,zk06:2181</va
lue>
</property>
<property>
  <name>yarn.resourcemanager.store.class</name>
  
<value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
</property>

<property>
  <name>yarn.resourcemanager.recovery.enabled</name>
  <value>true</value>
</property>
<property>
  <name>yarn.resourcemanager.zk-state-store.parent-path</name>
  <value>/yarn_ops02_rmstore</value>
</property>



> On Nov 3, 2015, at 6:10 AM, John Tipper <john_tip...@hotmail.com> wrote:
> 
> 
> 
> Does anyone have Samza working with resource manager in HA? If so, what do I 
> set yarn.resourcemanager.hostname to in yarn-site.xml?  Can anyone share a 
> working yarn-site.xml please?
> 
> 
> If I set it to the first of my RMs, the the job submission works ok if I 
> submit the job from that RM and the RM is the active one. If the RM machine 
> that I run the job submission from is not active, I get connection refused 
> errors on port 8032.  If I don't set it, I get errors where run-job.sh tries 
> to submit to 0.0.0.0:8032
> 
> 
> Many thanks,
> 
> 
> John

signature.asc
Description: Message signed with OpenPGP using GPGMail