nchammas commented on a change in pull request #27908:
URL: https://github.com/apache/spark/pull/27908#discussion_r474166293
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala
##########
@@ -342,6 +342,44 @@ abstract class Catalog {
createTable(tableName, source, schema, options)
}
+ /**
+ * Creates a table based on the dataset in a data source and a set of
options.
+ * Then, returns the corresponding DataFrame.
+ *
+ * @param tableName is either a qualified or unqualified name that
designates a table.
+ * If no database identifier is provided, it refers to a
table in
+ * the current database.
+ * @since 3.1.0
+ */
+ def createTable(
+ tableName: String,
+ source: String,
+ options: java.util.Map[String, String],
+ description: String): DataFrame = {
+ createTable(
+ tableName,
+ source = source,
+ options = options.asScala.toMap,
+ description = { if (description.isEmpty) None else Some(description) }
+ )
+ }
+
+ /**
+ * (Scala-specific)
+ * Creates a table based on the dataset in a data source and a set of
options.
+ * Then, returns the corresponding DataFrame.
+ *
+ * @param tableName is either a qualified or unqualified name that
designates a table.
+ * If no database identifier is provided, it refers to a
table in
+ * the current database.
+ * @since 3.1.0
+ */
+ def createTable(
+ tableName: String,
+ source: String,
+ options: Map[String, String],
+ description: Option[String]): DataFrame
Review comment:
Just to be clear about what I think I'm doing here (because I'm unsure
of myself and could use your confirmation):
At a high level, we have two versions of the `createTable()` APIs: one for
Scala, and one for Java and Python. Python calls the Java versions of the API
via Py4J.
On the Scala side, it's OK to use `Option[String]`. On the Java/Python side,
we need to use a plain `String` and then convert that to an `Option[String]`
when calling from the Java side into the Scala. That's what I'm currently doing.
Am I understanding correctly?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
