xiangfu0 commented on PR #18953:
URL: https://github.com/apache/pinot/pull/18953#issuecomment-4931926095
All review comments are addressed. Summary of the two follow-up commits:
**`4b7401f` — review fixes**
- **@xiangfu0's SQLite exact-fill bug** (the substantive one). Reproduced:
`0x0C` (OBJECT, payload size 0) followed by real data decoded to `{}` and
silently dropped the trailing bytes. `matches()` claims any payload whose first
byte carries the OBJECT nibble, so AUTO made this reachable for arbitrary input
— a corrupt stream message became an empty ingested row. `parse()` now enforces
SQLite's exact-fill rule, with regression tests at both the parser and
`JSONMessageDecoder` levels (and confirming a lone `0x0C` empty object still
parses).
- Malformed `{@code {}` Javadoc construct (verified with `mvn
javadoc:javadoc`).
- `fromConfig()` now preserves the `valueOf` failure as the exception cause.
- `testDefaultIsText` → `testUnsetFormatAutoDetectsText` (the default is
AUTO, not pinned TEXT).
- Test fixtures pinned to `StandardCharsets.UTF_8`.
- Coverage for previously untested SQLite branches: size descriptors 14/15
including oversized-`uint64` rejection, signed hex `INT5`, `FLOAT5` with a
leading `+`, the `\u` escape, the `TEXT5` escape extensions, and the
missing-value / non-text-label errors.
**`5289a94` — bounded, charset-safe decode diagnostic**
Also taking Copilot's low-confidence note on the exception message, which is
a real problem now that payloads can be binary: `new String(payload, offset,
length)` used the **platform default charset** and echoed the entire payload,
so a large Smile/CBOR/JSONB message was dumped into the log as unbounded
mojibake. It now renders at most the first 128 bytes — as UTF-8 when they
contain no control characters other than whitespace, as hex otherwise — and
always reports the true length. Text payloads keep their readable rendering;
binary payloads become a short hex prefix.
Module is at 65 tests, all passing; spotless / checkstyle / license clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
