huaxingao commented on a change in pull request #30154:
URL: https://github.com/apache/spark/pull/30154#discussion_r517695152
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCTableCatalog.scala
##########
@@ -117,14 +117,33 @@ class JDBCTableCatalog extends TableCatalog with Logging {
if (partitions.nonEmpty) {
throw new UnsupportedOperationException("Cannot create JDBC table with
partition")
}
- // TODO (SPARK-32405): Apply table options while creating tables in JDBC
Table Catalog
+
+ var tableOptions = options.parameters + (JDBCOptions.JDBC_TABLE_NAME ->
getTableName(ident))
+ var tableComment: String = ""
+ var tableProperties: String = ""
if (!properties.isEmpty) {
- logWarning("Cannot create JDBC table with properties, these properties
will be " +
- "ignored: " + properties.asScala.map { case (k, v) => s"$k=$v"
}.mkString("[", ", ", "]"))
+ properties.asScala.map {
+ case (k, v) => k match {
+ case "comment" => tableComment = v
+ case "provider" | "owner" | "location" => // provider, owner and
location can't be set.
+ case _ => tableProperties = tableProperties + " " + s"$k=$v"
+ }
+ }
}
- val writeOptions = new JdbcOptionsInWrite(
- options.parameters + (JDBCOptions.JDBC_TABLE_NAME ->
getTableName(ident)))
+ if (tableComment != "") {
+ tableOptions = tableOptions + (JDBCOptions.JDBC_TABLE_COMMENT ->
tableComment)
+ }
+ if (tableProperties != "") {
+ // table property is set in JDBC_CREATE_TABLE_OPTIONS, which will be
appended
+ // to CREATE TABLE statement.
+ // E.g., "CREATE TABLE t (name string) ENGINE=InnoDB DEFAULT
CHARSET=utf8"
Review comment:
I am thinking of removing the equal sign in between of the key and value
pairs so the create table statement will be ```CREATE TABLE t (name string)
ENGINE InnoDB DEFAULT CHARSET utf8```, so the key value pairs such as ```CREATE
TABLE t (name string) TABLESPACE mytspace``` will also work. The equal sign in
between of the key and value pairs in MySQL is optional anyway:
```
table_option: {
AUTO_INCREMENT [=] value
| AVG_ROW_LENGTH [=] value
| [DEFAULT] CHARACTER SET [=] charset_name
...
| ENGINE [=] engine_name
...
```
The `TBLPROPERTIES` seems to be a hive syntax and is not really used in the
traditional databases. I am not sure what properties in the traditional
databases should be supported in table creation. By removing the `=`, I think I
can at least support more properties here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
