linliu-code opened a new issue, #19708:
URL: https://github.com/apache/hudi/issues/19708
On a Merge-on-Read table using a global bloom or global simple index, a
`MERGE INTO` whose `UPDATE SET` does not assign the record key fails inside the
writer:
```
org.apache.hudi.exception.HoodieUpsertException: Failed to upsert for commit
time ...
at
org.apache.hudi.table.action.commit.BaseWriteHelper.write(BaseWriteHelper.java:84)
at
org.apache.hudi.table.action.deltacommit.SparkUpsertDeltaCommitActionExecutor.execute(SparkUpsertDeltaCommitActionExecutor.java:45)
Cause: org.apache.hudi.exception.HoodieKeyException:
recordKey value: "null" for field: "id" cannot be null or empty
at org.apache.hudi.keygen.KeyGenUtils.getRecordKey(KeyGenUtils.java:270)
at
org.apache.hudi.keygen.SimpleAvroKeyGenerator.getRecordKey(SimpleAvroKeyGenerator.java:50)
at
org.apache.spark.sql.hudi.command.SqlKeyGenerator.$anonfun$getPartitionPath$1(SqlKeyGenerator.scala:102)
```
Not assigning the record key is the ordinary shape for a partial update, so
this makes partial-update `MERGE INTO` unusable on that combination.
**To reproduce**
```sql
-- hoodie.index.type = GLOBAL_BLOOM
-- hoodie.bloom.index.update.partition.path = false
CREATE TABLE t (id BIGINT, name STRING, amount DOUBLE, ts BIGINT, dt STRING)
USING hudi PARTITIONED BY (dt)
TBLPROPERTIES (type = 'mor', primaryKey = 'id', preCombineField = 'ts');
INSERT INTO t VALUES (1, 'a', 10.0, 1, '2026-08-11');
MERGE INTO t AS t
USING (SELECT 1L AS id, 15.0 AS amount, 200L AS ts, '2026-08-11' AS dt) AS s
ON t.id = s.id
WHEN MATCHED THEN UPDATE SET t.amount = s.amount, t.ts = s.ts;
```
Expected: the row updates in place, with `name` and `dt` keeping their
existing values.
**Scope**
| Configuration | Result |
| --- | --- |
| MOR, `GLOBAL_BLOOM` or `GLOBAL_SIMPLE` | fails |
| MOR, `RECORD_INDEX` or `GLOBAL_RECORD_LEVEL_INDEX` | works |
| Copy-on-Write, any of the above | works |
| Same statement with the record key assigned | works |
The index types split because `mayContainDuplicateLookup` is `tableType ==
MERGE_ON_READ` for global bloom and global simple, while the record-index
implementations pass `false` and never reach the merge stage. Assigning the
record key works because it makes `areAllFieldsUpdated` true, which turns
partial updates off.
**Cause**
`SqlKeyGenerator#getPartitionPath(GenericRecord)` resolves the partition
path by way of `BaseKeyGenerator#getKey`, which is `new
HoodieKey(getRecordKey(record), getPartitionPath(record))`. Asking for a
partition path therefore also validates the record key. On the path above,
`HoodieIndexUtils#inferPartitionPath` asks for the partition path of a merged
record materialised against `WRITE_PARTIAL_UPDATE_SCHEMA`, which carries only
the columns named in `UPDATE SET`, so the record key is legitimately absent.
**Note**
Fixing this exposes a second, currently unreachable problem on the same
path, where the partial-update merged record is serialized against a mismatched
schema and raises `UnresolvedUnionException` from
`BaseAvroPayload#getRecordBytes`. That has a distinct cause in the payload path
and will be filed separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
