Re: Spark Streaming Kerberos Issue

Did you check that the security extensions are installed (JCE)? KhajaAsmath Mohammed schrieb am Mi., 22. Nov. 2017 um 19:36 Uhr: > [image: Inline image 1] > > This is what we are on. > > On Wed, Nov 22, 2017 at 12:33 PM, KhajaAsmath Mohammed < > mdkhajaasm...@gmail.com>

Re: Spark Streaming Kerberos Issue

[image: Inline image 1] This is what we are on. On Wed, Nov 22, 2017 at 12:33 PM, KhajaAsmath Mohammed < mdkhajaasm...@gmail.com> wrote: > We use oracle JDK. we are on unix. > > On Wed, Nov 22, 2017 at 12:31 PM, Georg Heiler > wrote: > >> Do you use oracle or open

Re: Spark Streaming Kerberos Issue

We use oracle JDK. we are on unix. On Wed, Nov 22, 2017 at 12:31 PM, Georg Heiler wrote: > Do you use oracle or open jdk? We recently had an issue with open jdk: > formerly, java Security extensions were installed by default - no longer so > on centos 7.3 > > Are

Re: Spark Streaming Kerberos Issue

Do you use oracle or open jdk? We recently had an issue with open jdk: formerly, java Security extensions were installed by default - no longer so on centos 7.3 Are these installed? KhajaAsmath Mohammed schrieb am Mi. 22. Nov. 2017 um 19:29: > I passed keytab, renewal

Re: Spark Streaming Kerberos Issue

I passed keytab, renewal is enabled by running the script every eight hours. User gets renewed by the script every eight hours. On Wed, Nov 22, 2017 at 12:27 PM, Georg Heiler wrote: > Did you pass a keytab? Is renewal enabled in your kdc? > KhajaAsmath Mohammed

Re: Spark Streaming Kerberos Issue

Did you pass a keytab? Is renewal enabled in your kdc? KhajaAsmath Mohammed schrieb am Mi. 22. Nov. 2017 um 19:25: > Hi, > > I have written spark stream job and job is running successfully for more > than 36 hours. After around 36 hours job gets failed with kerberos

Re: spark with kerberos

On 19 Oct 2016, at 00:18, Michael Segel > wrote: (Sorry sent reply via wrong account.. ) Steve, Kinda hijacking the thread, but I promise its still on topic to OP’s issue.. ;-) Usually you will end up having a local Kerberos set up

Re: spark with kerberos

(Sorry sent reply via wrong account.. ) Steve, Kinda hijacking the thread, but I promise its still on topic to OP’s issue.. ;-) Usually you will end up having a local Kerberos set up per cluster. So your machine accounts (hive, yarn, hbase, etc …) are going to be local to the cluster. So you

Re: spark with kerberos

On 17 Oct 2016, at 22:11, Michael Segel > wrote: @Steve you are going to have to explain what you mean by ‘turn Kerberos on’. Taken one way… it could mean making cluster B secure and running Kerberos and then you’d have to create

Re: spark with kerberos

On 13 Oct 2016, at 10:50, dbolshak > wrote: Hello community, We've a challenge and no ideas how to solve it. The problem, Say we have the following environment: 1. `cluster A`, the cluster does not use kerberos and we use it as a

Re: spark with kerberos

I think security has nothing to do with what API you use, spark sql or RDD API. Assuming you're running on yarn cluster (that is the only cluster manager supports Kerberos currently). Firstly you need to get Kerberos tgt in your local spark-submit process, after being authenticated by Kerberos,

Re: spark with kerberos

The problem happens when writting (reading works fine) rdd.saveAsNewAPIHadoopFile We use just RDD and HDFS, no other things. Spark 1.6.1 version. `Claster A` - CDH 5.7.1 `Cluster B` - vanilla hadoop 2.6.5 `Cluster C` - CDH 5.8.0 Best regards, Denis On 13 October 2016 at 13:06, ayan guha

Re: spark with kerberos

And a little more details on Spark version, hadoop version and distribution would also help... On Thu, Oct 13, 2016 at 9:05 PM, ayan guha wrote: > I think one point you need to mention is your target - HDFS, Hive or Hbase > (or something else) and which end points are used.

Re: spark with kerberos

I think one point you need to mention is your target - HDFS, Hive or Hbase (or something else) and which end points are used. On Thu, Oct 13, 2016 at 8:50 PM, dbolshak wrote: > Hello community, > > We've a challenge and no ideas how to solve it. > > The problem, > >

Re: Spark + Sentry + Kerberos don't add up?

Turns to be it is a Spark issue https://issues.apache.org/jira/browse/SPARK-13478 -- Ruslan Dautkhanov On Mon, Jan 18, 2016 at 4:25 PM, Ruslan Dautkhanov wrote: > Hi Romain, > > Thank you for your response. > > Adding Kerberos support might be as simple as >

Re: Spark + Sentry + Kerberos don't add up?

I took liberty and created a JIRA https://github.com/cloudera/livy/issues/36 Feel free to close it if doesn't belong to Livy project. I really don't know if this is a Spark or a Livy/Sentry problem. Any ideas for possible workarounds? Thank you. -- Ruslan Dautkhanov On Mon, Jan 18, 2016 at

Re: Spark + Sentry + Kerberos don't add up?

Hi Romain, Thank you for your response. Adding Kerberos support might be as simple as https://issues.cloudera.org/browse/LIVY-44 ? I.e. add Livy --principal and --keytab parameters to be passed to spark-submit. As a workaround I just did kinit (using hues' keytab) and then launched Livy Server.

Re: Spark + HBase + Kerberos

:46 PM To: Eric Walk Cc: user@spark.apache.org;Bill Busch Subject: Re: Spark + HBase + Kerberos Are hbase config / keytab files deployed on executor machines ? Consider adding -Dsun.security.krb5.debug=true for debug purpose. Cheers On Wed, Mar 18, 2015 at 11:39 AM, Eric Walk eric.w

Re: Spark + HBase + Kerberos

Are hbase config / keytab files deployed on executor machines ? Consider adding -Dsun.security.krb5.debug=true for debug purpose. Cheers On Wed, Mar 18, 2015 at 11:39 AM, Eric Walk eric.w...@perficient.com wrote: Having an issue connecting to HBase from a Spark container in a Secure

Re: [SPARK SQL] kerberos error when creating database from beeline/ThriftServer2

Which version of Spark and Hadoop are you using? Could you please provide the full stack trace of the exception? On Tue, Oct 28, 2014 at 5:48 AM, Du Li l...@yahoo-inc.com.invalid wrote: Hi, I was trying to set up Spark SQL on a private cluster. I configured a hive-site.xml under

Re: [SPARK SQL] kerberos error when creating database from beeline/ThriftServer2

-inc.com.invalidmailto:l...@yahoo-inc.com.invalid Cc: user@spark.apache.orgmailto:user@spark.apache.org user@spark.apache.orgmailto:user@spark.apache.org Subject: Re: [SPARK SQL] kerberos error when creating database from beeline/ThriftServer2 Which version of Spark and Hadoop are you using? Could you please

Re: [SPARK SQL] kerberos error when creating database from beeline/ThriftServer2

@spark.apache.org Subject: Re: [SPARK SQL] kerberos error when creating database from beeline/ThriftServer2 Which version of Spark and Hadoop are you using? Could you please provide the full stack trace of the exception? On Tue, Oct 28, 2014 at 5:48 AM, Du Li l...@yahoo-inc.com.invalidmailto:l

Re: [SPARK SQL] kerberos error when creating database from beeline/ThriftServer2

To: Cheng Lian lian.cs@gmail.commailto:lian.cs@gmail.com Cc: user@spark.apache.orgmailto:user@spark.apache.org user@spark.apache.orgmailto:user@spark.apache.org Subject: Re: [SPARK SQL] kerberos error when creating database from beeline/ThriftServer2 If I put all the jar files from my local