Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/4015#discussion_r28012573
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala ---
@@ -43,6 +43,68 @@ import org.apache.spark.util.Utils
import org.apache.spark.{Partition, SparkContext}
/**
+ * Currently we support the default dialect named "sql", associated with
the class
+ * [[DefaultSQLDialect]]
+ *
+ * And we can also provide custom SQL Dialect, for example in Spark SQL
CLI:
+ * {{{
+ *-- switch to "hiveql" dialect
+ * spark-sql>SET spark.sql.dialect=hiveql;
+ * spark-sql>SELECT * FROM src LIMIT 1;
+ *
+ *-- switch to "sql" dialect
+ * spark-sql>SET spark.sql.dialect=sql;
+ * spark-sql>SELECT * FROM src LIMIT 1;
+ *
+ *-- register the new SQL dialect
+ * spark-sql> SET spark.sql.dialect=com.xxx.xxx.SQL99Dialect;
+ * spark-sql> SELECT * FROM src LIMIT 1;
+ *
+ *-- register the non-exist SQL dialect
+ * spark-sql> SET spark.sql.dialect=NotExistedClass;
+ * spark-sql> SELECT * FROM src LIMIT 1;
+ *
+ *-- Exception will be thrown and switch to dialect
+ *-- "sql" (for SQLContext) or
+ *-- "hiveql" (for HiveContext)
+ * }}}
+ */
+@Experimental
+abstract class SQLDialect {
+ @transient
+ protected[sql] final val ddlParser = new DDLParser(parse)
+
+ /**
+ * We assume the DDLParser has higher priority than any of the other SQL
Parsers,
+ * We need to parse the DDL string first, if we can not get any result,
then will
+ * resort to the dialect parser. This method marked as final, which can
not be
+ * overwrite by its sub classes
+ */
+ protected[sql] final def apply(sqlText: String): LogicalPlan = {
+ ddlParser(sqlText, false).getOrElse(parse(sqlText))
+ }
+
+ // Will be called once after the instance created.
+ // this can be overwritten by subclasses for doing its own initialization
+ def initialize(sqlContext: SQLContext): SQLDialect = this
+
+ // this is the main function that will be implemented by sub classes.
+ protected def parse(sqlText: String): LogicalPlan
+}
+
+class DefaultSQLDialect extends SQLDialect {
--- End diff --
`private[spark]`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
