I am trying to use the HoodieSparkSQLWriter to upsert data from any
dataframe into a hoodie modeled table. Its creating everything correctly
but , i also want to save the checkpoint but i couldn't even though am
passing it as an argument.
inputDF.write()
.format("com.uber.hoodie")
.option(DataSourceWriteOptions.RECORDKEY_FIELD_OPT_KEY(), "_row_key")
.option(DataSourceWriteOptions.PARTITIONPATH_FIELD_OPT_KEY(), "partition")
.option(DataSourceWriteOptions.PRECOMBINE_FIELD_OPT_KEY(), "timestamp")
.option(HoodieWriteConfig.TABLE_NAME, tableName)
.option(DataSourceWriteOptions.COMMIT_METADATA_KEYPREFIX_OPT_KEY(),
checkpointstr)
.mode(SaveMode.Append)
.save(basePath);
am using the COMMIT_METADATA_KEYPREFIX_OPT_KEY() for inserting the
checkpoint while using the dataframe writer but i couldn't add the
checkpoint meta data in to the .hoodie meta data. Is there a way i can add
the checkpoint meta data while using the dataframe writer API?
