Hi Cheney,
I haven't heard of anybody deploying non-secure YARN on top of secure HDFS.
It's conceivable that you might be able to get work, but my guess is that
you'd run into some issues. Also, without authentication on in YARN, you
could be leaving your HDFS tokens exposed, which others could
Hi Sandy,
We are also going to grep data from a security enabled (with kerberos) HDFS
in our Spark application. Per you answer, we have to switch Spark on YARN
to achieve this.
We plan to deploy a different Hadoop cluster(with YARN) only to run Spark.
Is it necessary to deploy YARN with security e
Thanks a lot Marcelo and Sandy. I will try spark on yarn .
Xiaowei
From: Sandy Ryza [mailto:sandy.r...@cloudera.com]
Sent: Wednesday, July 09, 2014 4:20 AM
To: user@spark.apache.org
Subject: Re: error when spark access hdfs with Kerberos enable
That's correct. Only Spark on YARN sup
That's correct. Only Spark on YARN supports Kerberos.
-Sandy
On Tue, Jul 8, 2014 at 12:04 PM, Marcelo Vanzin wrote:
> Someone might be able to correct me if I'm wrong, but I don't believe
> standalone mode supports kerberos. You'd have to use Yarn for that.
>
> On Tue, Jul 8, 2014 at 1:40 AM,
Someone might be able to correct me if I'm wrong, but I don't believe
standalone mode supports kerberos. You'd have to use Yarn for that.
On Tue, Jul 8, 2014 at 1:40 AM, 许晓炜 wrote:
> Hi all,
>
>
>
> I encounter a strange issue when using spark 1.0 to access hdfs with
> Kerberos
>
> I just have on
Hi all,
I encounter a strange issue when using spark 1.0 to access hdfs with Kerberos
I just have one spark test node for spark and HADOOP_CONF_DIR is set to the
location containing the hdfs configuration files(hdfs-site.xml and
core-site.xml)
When I use spark-shell with local mode, the access t
