>From the programming guide: When working with Hive one must construct a HiveContext, which inherits > from SQLContext, and adds support for finding tables in in the MetaStore > and writing queries using HiveQL.
conf/ is a top level directory in the spark distribution that you downloaded. On Fri, Jul 25, 2014 at 2:35 PM, Sameer Tilak <ssti...@live.com> wrote: > Hi Jerry, > Thanks for your reply. I was following the steps in this programming > guide. It does not mention anything about creating HiveContext or HQL > explicitly. > > > > http://databricks.com/blog/2014/03/26/spark-sql-manipulating-structured-data-using-spark-2.html > > > - Users(userId INT, name String, email STRING, > age INT, latitude: DOUBLE, longitude: DOUBLE, > subscribed: BOOLEAN) > - Events(userId INT, action INT) > > Given the data stored in in these tables, one might want to build a model > that will predict which users are good targets for a new campaign, based on > users that are similar. > > // Data can easily be extracted from existing sources,// such as Apache > Hive.val trainingDataTable = sql(""" SELECT e.action u.age, > u.latitude, u.logitude FROM Users u JOIN Events e ON u.userId = > e.userId""") > > > > ------------------------------ > Date: Fri, 25 Jul 2014 17:27:17 -0400 > Subject: Re: Spark SQL and Hive tables > From: chiling...@gmail.com > To: user@spark.apache.org > > > Hi Sameer, > > Maybe this page will help you: > https://spark.apache.org/docs/latest/sql-programming-guide.html#hive-tables > > Best Regards, > > Jerry > > > > On Fri, Jul 25, 2014 at 5:25 PM, Sameer Tilak <ssti...@live.com> wrote: > > Hi All, > I am trying to load data from Hive tables using Spark SQL. I am using > spark-shell. Here is what I see: > > val trainingDataTable = sql("""SELECT prod.prod_num, demographics.gender, > demographics.birth_year, demographics.income_group FROM prod p JOIN > demographics d ON d.user_id = p.user_id""") > > 14/07/25 14:18:46 INFO Analyzer: Max iterations (2) reached for batch > MultiInstanceRelations > 14/07/25 14:18:46 INFO Analyzer: Max iterations (2) reached for batch > CaseInsensitiveAttributeReferences > java.lang.RuntimeException: Table Not Found: prod. > > I have these tables in hive. I used show tables command to confirm this. > Can someone please let me know how do I make them accessible here? > > >
