Re: Spark Thriftserver is failing for when submitting command from beeline

Looks like your problem is related to not setting up a hive.xml file properly.  The standard Spark distribution doesn't include a hive.xml template file in the conf directory.  You will have to create one by yourself.  Please refer to the Spark user doc and Hive metastore config guide for

Spark Thriftserver is failing for when submitting command from beeline

Hi Dev Environment details Hadoop 3.2 Hive 3.1 Spark 3.0.3 Cluster : Kerborized . 1) Hive server is running fine 2) Spark sql , sparkshell, spark submit everything is working as expected. 3) Connecting Hive through beeline is working fine (after kinit) beeline -u

Re: Spark Thriftserver on yarn, sql submit take long time.

case without explain, also take long time to submit 19/06/04 05:56:37 DEBUG SparkSQLOperationManager: Created Operation for > select count(*) from perf_as_reportads with > session=org.apache.hive.service.cli.session.HiveSessionImpl@1f30fc84, > runInBackground=true > 19/06/04 05:56:37 INFO

Spark Thriftserver on yarn, sql submit take long time.

Hi , Running thrift server on yarn. It's fast when beeline client send query to thrift server, but it take a while(about 90s) to submit to yarn cluster. >From Thrift server log: > *19/06/04 05:48:27* DEBUG SparkSQLOperationManager: Created Operation for > explain select count(*) from

Re:Customizing Spark ThriftServer

Spark ThriftServer is a spark application that possess thrift server. your code is a custom spark application. If you need some custome function beyond Spark ThriftServer, you can make your spark application contains HiveThriftServer2. At 2019-01-23 17:53:01, "Soheil Pourbafrani&qu

Customizing Spark ThriftServer

Hi, I want to create a thrift server that has some hive table predefined and listen on a port for the user query. Here is my code: val spark = SparkSession.builder() .config("hive.server2.thrift.port", "1") .config("spark.sql.hive.thriftServer.singleSession", "true")

Spark ThriftServer Error

Hi, When started thriftSever, got the following issue: 17/08/11 16:06:56 ERROR util.Utils: Uncaught exception in thread Thread-3 java.lang.NullPointerException at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$$anonfun$main$1.apply$mcV$sp(HiveThriftServer2.scala:85) at

Spark Thriftserver ERROR

Hi I tried to start spark-thrift server but get following error: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] java.io.IOException: javax.security.sasl.SaslException: GSS initiate

Re: Spark Thriftserver

Thank you Jeff. However, I am more looking for fine grained access control. For example: something like Ranger. Do you know if Spark thriftserver supported by Ranger or Sentry? Or something similar? Much appreciated On Wed, Mar 16, 2016 at 1:49 PM, Jeff Zhang <zjf...@gmail.com> wrote:

Re: Spark Thriftserver

.@gmail.com> wrote: > >> The spark thrift server allow you to run hive query in spark engine. It >> can be used as jdbc server. >> >> On Wed, Mar 16, 2016 at 10:42 AM, ayan guha <guha.a...@gmail.com> wrote: >> >>> Sorry to be dumb-head today, but what is the

Re: Spark Thriftserver

The spark thrift server allow you to run hive query in spark engine. It can be used as jdbc server. On Wed, Mar 16, 2016 at 10:42 AM, ayan guha <guha.a...@gmail.com> wrote: > Sorry to be dumb-head today, but what is the purpose of spark thriftserver > then? In other words, should

Re: Spark Thriftserver

spark thrift server is very similar with hive thrift server. You can use hive jdbc driver to access spark thrift server. AFAIK, all the features of hive thrift server are also available in spark thrift server. On Wed, Mar 16, 2016 at 8:39 AM, ayan guha wrote: > Hi All > >

Using different users with spark thriftserver

Hi, We are using spark thrift server as a hive replacement. One of the things we have with hive, is that different users can connect with their own usernames/passwords and get appropriate permissions. So on the same server, one user may have a query that will have permissions to run, while the

Spark Thriftserver exec insert sql got error on Hadoop federation

Hi all! My sql case is: insert overwrite table test1 select * From test; In the job end got move file error. I see hive-0.13.1 support for viewfs is not good. until hive-1.1.0+ How to upgrade the hive version for spark? Or how to fix the bug on org.spark-project.hive. My version: Spark version

Re: Spark-thriftserver Issue

Zhan specifying port fixed the port issue. Is it possible to specify the log directory while starting the spark thriftserver? Still getting this error even through the folder exists and everyone has permission to use that directory. drwxr-xr-x 2 root root 4096 Mar 24 19:04 spark

Re: Spark-thriftserver Issue

...@gmail.com wrote: Zhan specifying port fixed the port issue. Is it possible to specify the log directory while starting the spark thriftserver? Still getting this error even through the folder exists and everyone has permission to use that directory. drwxr-xr-x 2 root root 4096

Spark-thriftserver Issue

Hi, I am having issue starting spark-thriftserver. I'm running spark 1.3.with Hadoop 2.4.0. I would like to be able to change its port too so, I can hive hive-thriftserver as well as spark-thriftserver running at the same time. Starting sparkthrift server:- sudo ./start-thriftserver.sh --master

Re: Spark-thriftserver Issue

...@gmail.commailto:neilk...@gmail.com wrote: Hi, I am having issue starting spark-thriftserver. I'm running spark 1.3.with Hadoop 2.4.0. I would like to be able to change its port too so, I can hive hive-thriftserver as well as spark-thriftserver running at the same time. Starting sparkthrift server:- sudo ./start