Github user hvanhovell commented on a diff in the pull request:
https://github.com/apache/spark/pull/11048#discussion_r53568183
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala ---
@@ -62,6 +87,57 @@ private[sql] class SparkQl(conf: ParserConf =
SimpleParserConf()) extends Cataly
val tableIdent = extractTableIdent(nameParts)
RefreshTable(tableIdent)
+ case Token("TOK_CREATEDATABASE", Token(databaseName, Nil) ::
createDatabaseArgs) =>
+ val Seq(
+ allowExisting,
+ dbLocation,
+ databaseComment,
+ dbprops) = getClauses(Seq(
+ "TOK_IFNOTEXISTS",
+ "TOK_DATABASELOCATION",
+ "TOK_DATABASECOMMENT",
+ "TOK_DATABASEPROPERTIES"), createDatabaseArgs)
+
+ val location = dbLocation.map {
+ case Token("TOK_DATABASELOCATION", Token(loc, Nil) :: Nil) =>
unquoteString(loc)
+ }
+ val comment = databaseComment.map {
+ case Token("TOK_DATABASECOMMENT", Token(comment, Nil) :: Nil) =>
unquoteString(comment)
+ }
+ val props: Map[String, String] = dbprops.toSeq.flatMap {
+ case Token("TOK_DATABASEPROPERTIES", propList) =>
+ propList.flatMap(extractProps)
+ }.toMap
+
+ CreateDataBase(databaseName, allowExisting.isDefined, location,
comment, props)(node.source)
+
+ case Token("TOK_CREATEFUNCTION", func :: as :: createFuncArgs) =>
+ val funcName = func.map(x => unquoteString(x.text)).mkString(".")
+ val asName = unquoteString(as.text)
+ val Seq(
+ rList,
+ temp) = getClauses(Seq(
+ "TOK_RESOURCE_LIST",
+ "TOK_TEMPORARY"), createFuncArgs)
+
+ val resourcesMap: Map[String, String] = rList.toSeq.flatMap {
+ case Token("TOK_RESOURCE_LIST", resources) =>
+ resources.map {
+ case Token("TOK_RESOURCE_URI", rType :: Token(rPath, Nil) ::
Nil) =>
+ val resourceType = rType match {
+ case Token("TOK_JAR", Nil) => "jar"
+ case Token("TOK_FILE", Nil) => "file"
+ case Token("TOK_ARCHIVE", Nil) => "archive"
+ }
+ (resourceType, unquoteString(rPath))
+ }
+ }.toMap
+ CreateFunction(funcName, asName, resourcesMap,
temp.isDefined)(node.source)
+
+ case Token("TOK_ALTERTABLE", alterTableArgs) =>
+ AlterTableCommandParser(this).applyOrElse(node,
--- End diff --
The idea of the `PartialFunction` design was that we factor out all Parsing
functionality into a bunch of PartialFunctions each taking case of a specific
parsing area. This is nice because this allows us to compose a parser.
The code here just calls into the AlterTableCommandParser and treats it
like a regular function. This deviates from the design. There are few options
here:
- follow the proposed design;
- present a composable alternative and implement that;
- create a function in SparkQl which handles AlterTable commands, drop the
created infrastructure, and we will split the parsers in a followup PR.
I currently favor the third option.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
