[
https://issues.apache.org/jira/browse/HUDI-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lin Liu updated HUDI-7833:
--------------------------
Status: Open (was: In Progress)
> Validate that fg reader works with nested column as record key
> --------------------------------------------------------------
>
> Key: HUDI-7833
> URL: https://issues.apache.org/jira/browse/HUDI-7833
> Project: Apache Hudi
> Issue Type: Task
> Reporter: Jonathan Vexler
> Assignee: Lin Liu
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Ensure that fg reader works if the record key is a nested column
>
> Progress:
> Created a PR to reproduce the problem:
> [https://github.com/apache/hudi/pull/12253]
> From the PR, we turn on and off the fg reader, and run write operations
> (insert, update and delete), and read.
> We can see that with or without fg reader, the test would fail for map typed
> key.
> I did some investigation on the cause:
> in `BuiltinKeyGenerator.combineRecordKeyInternal`, the `UnsafeMapData`
> object's hash is returned, without caring the content of the map object.
> to fix it, we need to create a until function that deserialize these map
> objects, and generate the hash based on their content.
> We should also do this for all other nested data types.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
