Arush, Thank you will take a look at that approach in the morning. I sort of figured the answer to #1 was NO and that I would need to do 2 and 3 thanks for clarifying it for me.
-Todd On Tue, Feb 10, 2015 at 5:24 PM, Arush Kharbanda <ar...@sigmoidanalytics.com > wrote: > 1. Can the connector fetch or query schemaRDD's saved to Parquet or JSON > files? NO > 2. Do I need to do something to expose these via hive / metastore other > than creating a table in hive? Create a table in spark sql to expose via > spark sql > 3. Does the thriftserver need to be configured to expose these in some > fashion, sort of related to question 2 you would need to configure thrift > to read from the metastore you expect it read from - by default it reads > from metastore_db directory present in the directory used to launch the > thrift server. > On 11 Feb 2015 01:35, "Todd Nist" <tsind...@gmail.com> wrote: > >> Hi, >> >> I'm trying to understand how and what the Tableau connector to SparkSQL >> is able to access. My understanding is it needs to connect to the >> thriftserver and I am not sure how or if it exposes parquet, json, >> schemaRDDs, or does it only expose schemas defined in the metastore / hive. >> >> >> For example, I do the following from the spark-shell which generates a >> schemaRDD from a csv file and saves it as a JSON file as well as a parquet >> file. >> >> import *org.apache.sql.SQLContext >> *import com.databricks.spark.csv._ >> val sqlContext = new SQLContext(sc) >> val test = >> sqlContext.csfFile("/data/test.csv")test.toJSON().saveAsTextFile("/data/out") >> test.saveAsParquetFile("/data/out") >> >> When I connect from Tableau, the only thing I see is the "default" schema >> and nothing in the tables section. >> >> So my questions are: >> >> 1. Can the connector fetch or query schemaRDD's saved to Parquet or JSON >> files? >> 2. Do I need to do something to expose these via hive / metastore other >> than creating a table in hive? >> 3. Does the thriftserver need to be configured to expose these in some >> fashion, sort of related to question 2. >> >> TIA for the assistance. >> >> -Todd >> >