How to use Spark scala custom UDF in spark sql CLI or beeline client

[pooja mehta]

Hi,

How to Use Spark scala custom UDF in spark sql CLI or Beeline client.
with sqlContext we can register a UDF like this:
 sqlContext.udf.register("sample_fn", sample_fn _ )
What is the way to use UDF in Spark sql CLI or beeline client.

Thanks
Pooja

Spark sql dataframe

[pooja mehta]

Hi,

Want to add a metadata field to StructField case class in spark.

case class StructField(name: String)

And how to carry over the metadata in query execution.

[no subject]

[pooja mehta]

Hi,

Want to add a metadata field to StructField case class in spark.

case class StructField(name: String)

And how to carry over the metadata in query execution.

Spark Thrift Server in CDH 5.3

[pooja mehta]

Hi,

How do I start Spark Thrift Server with cloudera CDH 5.3?

Thanks.

[no subject]

[pooja mehta]

Hi,

How to use scala UDF with the help of Beeline client.
With the help of spark shell, we register our UDF like this:-
sqlcontext.udf.register().
What is the way to use UDF in beeline client.

Thanks
Pooja

Fwd: Beeline - Spark thrift server user retrieval Issue

[pooja mehta]

-- Forwarded message --
From: pooja mehta <sparkrad...@gmail.com>
Date: Thu, Jun 2, 2016 at 1:25 PM
Subject: Fwd: Beeline - Spark thrift server user retrieval Issue
To: user-subscr...@spark.apache.org



-- Forwarded message --
From: pooja mehta <sparkrad...@gmail.com>
Date: Thu, Jun 2, 2016 at 1:24 PM
Subject: Beeline - Spark thrift server user retrieval Issue
To: user-i...@spark.apache.org, user-...@spark.apache.org


Hi,

I am using spark Beeline client and When i logged in as newuser(Lets
suppose user2) and connect to port 10015(Using spark thrift server),
And use
 userName =UserGroupInformation.getCurrentUser().getShortUserName();
It always gives hive user,not user2.

Is there any way to get user2?

While when i am connecting to port 1(Hive thrift server ) via spark
beeline ,it gives user2.