nsivabalan commented on a change in pull request #1834:
URL: https://github.com/apache/hudi/pull/1834#discussion_r468656221
##########
File path: hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala
##########
@@ -105,6 +104,22 @@ private[hudi] object HoodieSparkSqlWriter {
} else {
// Handle various save modes
handleSaveModes(mode, basePath, tableConfig, tblName, operation, fs)
+ // Create the table if not present
+ if (!tableExists) {
+ val tableMetaClient =
HoodieTableMetaClient.initTableType(sparkContext.hadoopConfiguration, path.get,
+ HoodieTableType.valueOf(tableType), tblName, "archived",
parameters(PAYLOAD_CLASS_OPT_KEY),
+ null.asInstanceOf[String])
+ tableConfig = tableMetaClient.getTableConfig
+ }
+
+ // short-circuit if bulk_insert via row is enabled.
+ // scalastyle:off
+ if (operation.equalsIgnoreCase(BULK_INSERT_DATASET_OPERATION_OPT_VAL)) {
+ val (success, commitTime: common.util.Option[String]) =
bulkInsertAsRow(sqlContext, parameters, df, tblName,
+
basePath, path, instantTime)
+ return (success, commitTime, common.util.Option.of(""),
hoodieWriteClient.orNull, tableConfig)
Review comment:
nit: can you make the 3rd arg Option.empty. when I put up the PR, I got
compilation issues and hence returned empty string. I tested Option.empty
locally with latest change and compilation seems to succeed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
