: Andrew Lee
To: "user@spark.apache.org"
Date: 05/06/2014 02:26 AM
Subject: RE: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
Hi Jacob,
I agree, we need to address both driver and workers bidirectionally.
If the subnet i
24 can allow SSH 22 login (or port forwarding) onto the Spark Driver
machine to launch shell or submit spark jobs.
Subject: RE: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
To: user@spark.apache.org
From: jeis...@us.ibm.com
Date: Mon, 5 May 2014 12:40:
(512) 286-6075
From: Andrew Lee
To: "user@spark.apache.org"
Date: 05/04/2014 09:57 PM
Subject: RE: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
Hi Jacob,
Taking both concerns into account, I'm actually th
y, it is
running ok with iptables off, but still need to figure out how to
product-ionize the security part.
Subject: RE: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
To: user@spark.apache.org
From: jeis...@us.ibm.com
Date: Fri, 2 May 2014 16:07:50
m - (512) 286-6075
From: Andrew Lee
To: "user@spark.apache.org"
Date: 05/02/2014 03:15 PM
Subject: RE: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
Hi Yana,
I did. I configured the the port in spark-env.sh, the pr
y 2014 14:49:23 -0400
Subject: Re: spark-shell driver interacting with Workers in YARN mode -
firewall blocking communication
From: yana.kadiy...@gmail.com
To: user@spark.apache.org
I think what you want to do is set spark.driver.port to a fixed port.
On Fri, May 2, 2014 at 1:52 PM, Andrew Lee wr
I think what you want to do is set spark.driver.port to a fixed port.
On Fri, May 2, 2014 at 1:52 PM, Andrew Lee wrote:
> Hi All,
>
> I encountered this problem when the firewall is enabled between the
> spark-shell and the Workers.
>
> When I launch spark-shell in yarn-client mode, I notice th
