Jackie-Jiang commented on code in PR #18976:
URL: https://github.com/apache/pinot/pull/18976#discussion_r4498969993
##########
pinot-plugins/pinot-input-format/pinot-json/src/main/java/org/apache/pinot/plugin/inputformat/json/JSONRecordReader.java:
##########
@@ -49,7 +49,7 @@ private void init()
throws IOException {
_inputStream = RecordReaderUtils.getBufferedInputStream(_dataFile);
try {
- _iterator = JsonUtils.DEFAULT_READER.forType(new
TypeReference<Map<String, Object>>() {
+ _iterator = JsonUtils.READER_WITH_BIG_DECIMAL.forType(new
TypeReference<Map<String, Object>>() {
}).readValues(_inputStream);
Review Comment:
[P1] Handle mixed numeric arrays when converting to MV BIG_DECIMAL
Batch ingestion of `{"v":[1.5,2]}` into an MV BIG_DECIMAL column previously
succeeded, but now throws `ClassCastException`. The decoded array contains a
BigDecimal followed by an Integer. `DataTypeTransformerUtils.transformValue()`
infers `BIG_DECIMAL_ARRAY` from the first element, and
`PinotDataType.toBigDecimalArray()` applies `BIG_DECIMAL.toBigDecimal()` to
every element. Its direct cast fails on the Integer second element. Before this
PR, the inferred source was `DOUBLE_ARRAY`, whose `toBigDecimal()` uses `new
BigDecimal(value.toString())` and accepts both numbers.
MV BIG_DECIMAL is a supported offline ingestion path, including dictionary
and raw columns. Depending on error handling, this change fails ingestion or
nulls the column.
Please add per-element source-type dispatch or fallback to
`toBigDecimalArray()`, analogous to the STRING fix, and regression coverage for
converting `{BigDecimal("1.5"), 2}` to `BIG_DECIMAL_ARRAY`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]