Re: Hive on Spark application will be submited more times when the queue resources is not enough.

Hi Jone, Thanks for reporting the problem. When you say there is no enough resource, do you mean that you cannot launch Yarn application masters? I feel that we should error out right way if the application cannot be submitted. Any attempt of resubmitted seems problematic. I'm not sure if there i

About Permanent Functions design

Hi, I found after hive 0.13. user can register Permanent Functions。this is a very good feature! but i want to know why the functions name must bind for the dbname,instead of provide a way to allow user register global function. I would like to know what the motivation of this design? thanks.

Re: Hive on Spark application will be submited more times when the queue resources is not enough.

*It seems that the submit number depend on stage of the query.* *This query include three stages.* If queue resources is still *not enough after submit threee applications,** Hive client will close.* *"**Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException

Re: Hive Support for Unicode languages

Presuming you are using some utility like Cygwin/Putty etc. to access Hive CLI , you would need to configure the settings of this application for supporting extended charset display. Easy/Quick option to verify the support will be by using some data browser application (e.g. Talend ) to sample dat

Re: Metastore and SerDe jars

I think this is still an outstanding issue. Couple of (still open) Jiras relating to solutions for this: https://issues.apache.org/jira/browse/HIVE-9252? https://issues.apache.org/jira/browse/HIVE-6343 From: Andy Srine Sent: Wednesday, December 09, 2015 1:33

Re: Create table from ORC or Parquet file?

E.g. in Spark SQL I can create temporary table from ORC, Parquet or json files without specifying column names and types val myDf = sqlContext.read.format("orc").load("s3n://alex/test/mytable_orc") myDf.printSchema root |-- id: string (nullable = true) |-- name: string (nullable = true) |-- rc

Re: Hive Support for Unicode languages

Any update ? On 12/5/2015 2:10 PM, Mahender Sarangam wrote: Its Windows Server 2012 OS. > From: jornfra...@gmail.com > Subject: Re: Hive Support for Unicode languages > Date: Fri, 4 Dec 2015 13:19:00 +0100 > To: user@hive.apache.org > > What operating system are you using? > > > On 04 Dec 2015,

Re: Create table from ORC or Parquet file?

So your use case is that you already have the ORC files and you want a table that can read those files without specifying the columns in the table? Obviously without the columns being specified Hive wouldn't be able to write to that table, so I assume you only care about reading it. Is that right?

Issues running 0.14 hive JDBC driver on jetty 9.1.1.20140108

Hi, I'm having a lot of difficulty using the hive JDBC driver running in a jetty container. The hive JDBC driver pulls in a much earlier version of jetty-all, which causes a failure when the jetty container starts up: java.lang.IllegalArgumentException: Object of class 'org.eclipse.jetty.webap

Re: Hive on Spark application will be submited more times when the queue resources is not enough.

> > But in some cases all of the applications will fail which caused > by SparkContext did not initialize after waiting for 15 ms. > See attchment (hive.spark.client.server.connect.timeout is set to 5min). *The error log is different from original mail* Container: container_1448873753366_11

Re: Hive on Spark application will be submited more times when the queue resources is not enough.

Hive version is 1.2.1 Spark version is 1.4.1 Hadoop version is 2.5.1 The application_1448873753366_121062 will success in the above mail. But in some cases all of the applications will fail which caused by SparkContext did not initialize after waiting for 15 ms. See attchment (hive.spark.clie

Re: Create table from ORC or Parquet file?

Hi Stephen, Can you share example of how are you doing it ? Would really appreciate your help. As I am also stuck in this scenario . Thanks, Divya On Dec 8, 2015 11:17 PM, "Stephen Bly" wrote: > I am working on a similar problem — creating a Hive table from Parquet > data and using the embedded

Hive on Spark application will be submited more times when the queue resources is not enough.

*Hi, Xuefu:* *See attachment 1* *When the queue resources is not enough.* *The application application_1448873753366_121022 will pending.* *Two minutes later, the application application_1448873753366_121055 will be submited and pending.* *And then application_1448873753366_121062.* *See attachme

getting error while persisting in hive

Hi, I am using spark 1.4.1 . I am getting error when persisting spark dataframe output to hive > scala> > df.select("name","age").write().format("com.databricks.spark.csv").mode(SaveMode.Append).saveAsTable("PersonHiveTable"); > :39: error: org.apache.spark.sql.DataFrameWriter does not take > para

Metastore and SerDe jars

Team, I know I can do a "ADD JAR" on the Hive CLI to use SerDe and UDF jars. If I am using the metastore thrift API, is there a way to add/register SerDe jars via the API? Basically, whats the metastore API equivalent of CLI's "ADD JAR "? I tried setting the hive.aux.jars.path property in the hive