@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens
dari...@blackberry.commailto:dari...@blackberry.com wrote:
Would there be any way to have the task instances in the slaves call the UGI
login with a principal/keytab
, thanks everyone.
From: Steve Loughran [mailto:ste...@hortonworks.com]
Sent: Monday, June 29, 2015 10:32 AM
To: Dave Ariens
Cc: Tim Chen; Marcelo Vanzin; Olivier Girardot; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On 29 Jun 2015, at 14:18, Dave
On 29 Jun 2015, at 14:18, Dave Ariens
dari...@blackberry.commailto:dari...@blackberry.com wrote:
I'd like to toss out another idea that doesn't involve a complete end-to-end
Kerberos implementation. Essentially, have the driver authenticate to
Kerberos, instantiate a Hadoop file system, and
: Accessing Kerberos Secured HDFS Resources from Spark on
Mesos
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens dari...@blackberry.com
wrote:
Would there be any way to have the task instances in the slaves call
the UGI login with a principal/keytab provided to the driver?
That would only work
: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens
dari...@blackberry.commailto:dari...@blackberry.com wrote:
Would there be any way to have the task instances in the slaves call the UGI
login with a principal/keytab provided to the driver
*Sent: *Friday, June 26, 2015 6:20 PM
*To: *Dave Ariens
*Cc: *Tim Chen; Olivier Girardot; user@spark.apache.org
*Subject: *Re: Accessing Kerberos Secured HDFS Resources from Spark on
Mesos
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens dari...@blackberry.com
wrote:
Would there be any way
I understand that Kerberos support for accessing Hadoop resources in Spark only
works when running Spark on YARN. However, I'd really like to hack something
together for Spark on Mesos running alongside a secured Hadoop cluster. My
simplified appplication (gist:
Hi Dave,
I don't understand Keeberos much but if you know the exact steps that needs to
happen I can see how we can make that happen with the Spark framework.
Tim
On Jun 26, 2015, at 8:49 AM, Dave Ariens dari...@blackberry.com wrote:
I understand that Kerberos support for accessing
Mesos do support running containers as specific users passed to it.
Thanks for chiming in, what else does YARN do with Kerberos besides keytab
file and user?
Tim
On Fri, Jun 26, 2015 at 1:20 PM, Marcelo Vanzin van...@cloudera.com wrote:
On Fri, Jun 26, 2015 at 1:13 PM, Tim Chen
On Fri, Jun 26, 2015 at 2:08 PM, Tim Chen t...@mesosphere.io wrote:
Mesos do support running containers as specific users passed to it.
Thanks for chiming in, what else does YARN do with Kerberos besides keytab
file and user?
The basic things I'd expect from a system to properly support
:* user@spark.apache.org
*Subject:* Re: Accessing Kerberos Secured HDFS Resources from Spark on
Mesos
Hi Dave,
I don't understand Keeberos much but if you know the exact steps that
needs to happen I can see how we can make that happen with the Spark
framework.
Tim
On Jun 26, 2015
On Fri, Jun 26, 2015 at 1:13 PM, Tim Chen t...@mesosphere.io wrote:
So correct me if I'm wrong, sounds like all you need is a principal user
name and also a keytab file downloaded right?
I'm not familiar with Mesos so don't know what kinds of features it has,
but at the very least it would
Chen
Cc: Olivier Girardot; Dave Ariens; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On Fri, Jun 26, 2015 at 1:13 PM, Tim Chen
t...@mesosphere.iomailto:t...@mesosphere.io wrote:
So correct me if I'm wrong, sounds like all you need is a principal
help would be appreciated!
From: Timothy Chen [mailto:t...@mesosphere.io]
Sent: Friday, June 26, 2015 12:50 PM
To: Dave Ariens
Cc: user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
Hi Dave,
I don't understand Keeberos much but if you know the exact
Ariens
*Cc:* user@spark.apache.org
*Subject:* Re: Accessing Kerberos Secured HDFS Resources from Spark on
Mesos
Hi Dave,
I don't understand Keeberos much but if you know the exact steps that
needs to happen I can see how we can make that happen with the Spark
framework.
Tim
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens dari...@blackberry.com wrote:
Would there be any way to have the task instances in the slaves call the
UGI login with a principal/keytab provided to the driver?
That would only work with a very small number of executors. If you have
many login
PM
To: Dave Ariens
Cc: Tim Chen; Olivier Girardot; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens
dari...@blackberry.commailto:dari...@blackberry.com wrote:
Would there be any way to have the task
. You can check the Hadoop
sources for details. Not sure if there's another way.
*From: *Marcelo Vanzin
*Sent: *Friday, June 26, 2015 6:20 PM
*To: *Dave Ariens
*Cc: *Tim Chen; Olivier Girardot; user@spark.apache.org
*Subject: *Re: Accessing Kerberos Secured HDFS Resources from Spark
in the slaves call
the UGI login with a principal/keytab provided to the driver?
From: Marcelo Vanzin
Sent: Friday, June 26, 2015 5:28 PM
To: Tim Chen
Cc: Olivier Girardot; Dave Ariens; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
On Fri, Jun 26
19 matches
Mail list logo
