Hi, Spark users:
We currently are using Spark 1.2.2 + Hive 0.12 + Hadoop 2.2.0 on our production
cluster, which has 42 data/task nodes.
There is one dataset stored as Avro files about 3T. Our business has a complex
query running for the dataset, which is stored in nest structure with Array of
...@databricks.com
Date: Fri, 7 Aug 2015 11:32:21 -0700
Subject: Re: Spark SQL query AVRO file
To: java8...@hotmail.com
CC: user@spark.apache.org
Have you considered trying Spark SQL's native support for avro data?
https://github.com/databricks/spark-avro
On Fri, Aug 7, 2015 at 11:30 AM, java8964 java8
--
From: mich...@databricks.com
Date: Fri, 7 Aug 2015 11:32:21 -0700
Subject: Re: Spark SQL query AVRO file
To: java8...@hotmail.com
CC: user@spark.apache.org
Have you considered trying Spark SQL's native support for avro data?
https://github.com/databricks/spark-avro
On Fri, Aug 7, 2015
Good to know that.
Let me research it and give it a try.
Thanks
Yong
From: mich...@databricks.com
Date: Fri, 7 Aug 2015 11:44:48 -0700
Subject: Re: Spark SQL query AVRO file
To: java8...@hotmail.com
CC: user@spark.apache.org
You can register your data as a table using this library and then query
