featzhang commented on PR #27897:
URL: https://github.com/apache/flink/pull/27897#issuecomment-4229758127
Thanks for the thorough review @spuru9 and @rmetzger!
Here's what was addressed in this update:
1. **Commit message**: Updated to include the Jira ticket ID:
`[FLINK-39401][formats]`.
2. **Round-trip compatibility fix** (serializer/deserializer
incompatibility): The deserializer now strips the single trailing empty string
produced when a message ends with the delimiter. This makes serialization and
deserialization fully round-trip compatible — `serialize("hello") → "hello\n" →
deserialize → ["hello"]` (1 row, not 2). Added
`testDeserializeTrailingDelimiter_noExtraRow` and
`testRoundTrip_serializeThenDeserialize` tests to verify this.
3. **Pre-compiled Pattern**: `Pattern.quote(lineDelimiter)` is now
pre-compiled once in the constructor as a `Pattern lineDelimiterPattern` field,
eliminating repeated compilation on every message.
4. **Pre-computed delimiterBytes**: `delimiterBytes` is now computed once in
the constructor and stored as a field, eliminating repeated allocation on every
row serialization.
5. **Documentation**: Added `raw.line-delimiter` option to both
`docs/content/docs/connectors/table/formats/raw.md` and
`docs/content.zh/docs/connectors/table/formats/raw.md`, including a note on
round-trip compatibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
