linliu-code opened a new pull request, #19454:
URL: https://github.com/apache/hudi/pull/19454
### Change Logs
`HoodieLogBlock.getLogMetadata()` read each metadata entry with `new
String(byte[])`, which applies the **platform default charset**. The write
side, `getLogMetadataBytes()`, serializes those same values with
`StringUtils.getUTF8Bytes()`. The two sides therefore disagree on any JVM whose
default charset is not UTF-8.
```java
// before
metadata.put(typeMapper.apply(metadataEntryIndex), new
String(metadataEntry));
// after
metadata.put(typeMapper.apply(metadataEntryIndex),
fromUTF8Bytes(metadataEntry));
```
The visible consequence is a corrupted header/footer value on read. When
that value is the writer schema and it contains non-ASCII field names, the
corrupted string then fails Avro parsing with `Illegal initial character`.
`StringUtils.fromUTF8Bytes` already lives in the same class this file
imports `getUTF8Bytes` from, so no new dependency is introduced.
Also adds a round-trip test asserting that a schema containing a non-ASCII
field name survives `getHeaderMetadataBytes` → `getHeaderMetadata` and still
parses as Avro.
### Impact
Read path only, for log block header/footer metadata.
Correct on any JVM whose default charset is already UTF-8 — behaviour there
is unchanged. On a JVM with a non-UTF-8 default charset, non-ASCII header
values now round-trip instead of being corrupted.
One case worth calling out explicitly: if a table's log blocks were
*written* by a JVM whose default charset was not UTF-8, those header bytes are
not valid UTF-8 on disk, and this change makes them decode differently than
before. That data was already being written inconsistently with
`getLogMetadataBytes()`'s contract, so I believe aligning the read with the
documented write encoding is correct — but I'd appreciate a reviewer's view on
whether any compatibility shim is wanted.
### Risk level: low
Two-line change on the read side, plus a test.
**On the test's scope, stated plainly:** the old and new code agree whenever
the JVM default charset is already UTF-8, so this test passes both before and
after the change on a UTF-8 JVM. It documents the invariant rather than
reproducing the failure. Reproducing the original failure requires a non-UTF-8
default charset — under `-Dfile.encoding=ISO-8859-1`, `new
String("名字".getBytes(UTF_8))` demonstrably does not round-trip. I was not able
to get surefire to honour that override in this module (the build uses
`<argLine>@{argLine}</argLine}` with the `argLine` property redefined per
profile), so the negative case is verified outside the test harness rather than
by a failing test.
Verified: `hudi-common` compiles, and the new test passes (`Tests run: 1,
Failures: 0, Errors: 0`).
### Documentation Update
None needed.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Change Logs and Impact were stated clearly
- [x] Adequate tests were added if applicable
- [x] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
