Re: what is the roadmap for Spark SQL dialect in the coming releases?
Thanks Michael. A clarification. So the HQL dialect provided by HiveContext, does it use catalyst optimizer? I though HiveContext is only related to Hive integration in Spark! Would be grateful if you could clarify this cheers On Sun, Jan 25, 2015 at 1:23 AM, Michael Armbrust mich...@databricks.com wrote: I generally recommend people use the HQL dialect provided by the HiveContext when possible: http://spark.apache.org/docs/latest/sql-programming-guide.html#getting-started I'll also note that this is distinct from the Hive on Spark project, which is based on the Hive query optimizer / execution engine instead of the catalyst optimizer that is shipped with Spark. On Thu, Jan 22, 2015 at 3:12 AM, Niranda Perera niranda.per...@gmail.com wrote: Hi, would like to know if there is an update on this? rgds On Mon, Jan 12, 2015 at 10:44 AM, Niranda Perera niranda.per...@gmail.com wrote: Hi, I found out that SparkSQL supports only a relatively small subset of SQL dialect currently. I would like to know the roadmap for the coming releases. And, are you focusing more on popularizing the 'Hive on Spark' SQL dialect or the Spark SQL dialect? Rgds -- Niranda -- Niranda -- Niranda
Re: what is the roadmap for Spark SQL dialect in the coming releases?
Yeah, the HiveContext is just a SQLContext that is extended with HQL, access to a metastore, hive UDFs and hive serdes. The query execution however is identical to a SQLContext. On Sun, Jan 25, 2015 at 7:24 AM, Niranda Perera niranda.per...@gmail.com wrote: Thanks Michael. A clarification. So the HQL dialect provided by HiveContext, does it use catalyst optimizer? I though HiveContext is only related to Hive integration in Spark! Would be grateful if you could clarify this cheers On Sun, Jan 25, 2015 at 1:23 AM, Michael Armbrust mich...@databricks.com wrote: I generally recommend people use the HQL dialect provided by the HiveContext when possible: http://spark.apache.org/docs/latest/sql-programming-guide.html#getting-started I'll also note that this is distinct from the Hive on Spark project, which is based on the Hive query optimizer / execution engine instead of the catalyst optimizer that is shipped with Spark. On Thu, Jan 22, 2015 at 3:12 AM, Niranda Perera niranda.per...@gmail.com wrote: Hi, would like to know if there is an update on this? rgds On Mon, Jan 12, 2015 at 10:44 AM, Niranda Perera niranda.per...@gmail.com wrote: Hi, I found out that SparkSQL supports only a relatively small subset of SQL dialect currently. I would like to know the roadmap for the coming releases. And, are you focusing more on popularizing the 'Hive on Spark' SQL dialect or the Spark SQL dialect? Rgds -- Niranda -- Niranda -- Niranda
Re: what is the roadmap for Spark SQL dialect in the coming releases?
I generally recommend people use the HQL dialect provided by the HiveContext when possible: http://spark.apache.org/docs/latest/sql-programming-guide.html#getting-started I'll also note that this is distinct from the Hive on Spark project, which is based on the Hive query optimizer / execution engine instead of the catalyst optimizer that is shipped with Spark. On Thu, Jan 22, 2015 at 3:12 AM, Niranda Perera niranda.per...@gmail.com wrote: Hi, would like to know if there is an update on this? rgds On Mon, Jan 12, 2015 at 10:44 AM, Niranda Perera niranda.per...@gmail.com wrote: Hi, I found out that SparkSQL supports only a relatively small subset of SQL dialect currently. I would like to know the roadmap for the coming releases. And, are you focusing more on popularizing the 'Hive on Spark' SQL dialect or the Spark SQL dialect? Rgds -- Niranda -- Niranda
Re: what is the roadmap for Spark SQL dialect in the coming releases?
Hi, would like to know if there is an update on this? rgds On Mon, Jan 12, 2015 at 10:44 AM, Niranda Perera niranda.per...@gmail.com wrote: Hi, I found out that SparkSQL supports only a relatively small subset of SQL dialect currently. I would like to know the roadmap for the coming releases. And, are you focusing more on popularizing the 'Hive on Spark' SQL dialect or the Spark SQL dialect? Rgds -- Niranda -- Niranda
what is the roadmap for Spark SQL dialect in the coming releases?
Hi, I found out that SparkSQL supports only a relatively small subset of SQL dialect currently. I would like to know the roadmap for the coming releases. And, are you focusing more on popularizing the 'Hive on Spark' SQL dialect or the Spark SQL dialect? Rgds -- Niranda
