noobarcitect commented on issue #1586:
URL: https://github.com/apache/hudi/issues/1586#issuecomment-683269914
@bvaradar
is this issue resolved in any of Hudi release ?
I am currently using 0.6.0 with scala 2.11 and previously tried with
0.5.2-incubating/0.5.3 + Scala 2.11 as well the result is same Below is the
description of the spark job that I wrote.
My usecase is somewhat same where I am using 2 columns of a dataset to make
an upsert. but the comma separated values don't work. Below is the code I am
using to make an upsert in a dataset where albumId and trackId are 2 keys which
identifies a unique record :
albumDf.write
.format("hudi")
.option(DataSourceWriteOptions.TABLE_TYPE_OPT_KEY,
DataSourceWriteOptions.COW_TABLE_TYPE_OPT_VAL)
.option(DataSourceWriteOptions.RECORDKEY_FIELD_OPT_KEY, "albumId,
trackId")
.option(DataSourceWriteOptions.PRECOMBINE_FIELD_OPT_KEY, combineKey)
.option(HoodieWriteConfig.TABLE_NAME, tableName)
.option(DataSourceWriteOptions.OPERATION_OPT_KEY,
DataSourceWriteOptions.UPSERT_OPERATION_OPT_VAL)
.option("hoodie.upsert.shuffle.parallelism", "2")
.mode(SaveMode.Append)
.save(s"$basePath/$tableName/")
And below is the error that I get :
... 5 more
Caused by: org.apache.hudi.exception.HoodieKeyException: recordKey value:
"null" for field: "albumId, trackId" cannot be null or empty.
at org.apache.hudi.keygen.SimpleKeyGenerator.getKe
Can you please help ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
