rangareddy commented on issue #14443:
URL: https://github.com/apache/hudi/issues/14443#issuecomment-4902430294
This issue was reviewed as part of the JIRA-migrated backlog triage, and I
reproduced it to confirm the fix.
Root cause: when a source JSON field's value is a JSON object but the target
schema types that field as `string`, `MercifulJsonConverter` (the JSON->Avro
converter used by the Hudi streamer) wrote the value via the parsed object's
`toString()`. That object is a Java `Map`, so `toString()` renders `{a=1, b=x}`
instead of the original JSON `{"a":1,"b":"x"}`.
This was fixed by #11825 (HUDI-8125, commit f64ff33), which changed the
string handler to pass through String values unchanged and serialize non-String
values with Jackson `ObjectMapper.writeValueAsString` (`StringProcessor` in
hudi-common `org/apache/hudi/avro/MercifulJsonConverter.java`). First released
in 1.0.0.
Verified with a minimal `MercifulJsonConverter.convert()` test (field
`payload` typed `string`, value `{"a":1,"b":"x","nested":{"k":"v"}}`):
- Hudi 0.15.0 -> `payload = {a=1, b=x, nested={k=v}}` (the bug)
- Hudi 1.0.2 -> `payload = {"a":1,"b":"x","nested":{"k":"v"}}` (fixed,
round-trips the input)
Note: this issue's fixVersion (0.16.0) was never released; the fix landed on
the 1.0.x line via HUDI-8125.
If you're still hitting this on 1.0.0+, please reopen with repro steps.
Closing as fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
