yashmayya opened a new pull request, #16885:
URL: https://github.com/apache/pinot/pull/16885
- A simple `CAST(stringCol AS TIMESTAMP)` currently fails for null values
with an error like:
```
Caused by: java.lang.IllegalArgumentException: Invalid timestamp: 'null'
at
org.apache.pinot.spi.utils.TimestampUtils.toMillisSinceEpoch(TimestampUtils.java:132)
at
org.apache.pinot.spi.utils.ArrayCopyUtils.copyToTimestamp(ArrayCopyUtils.java:283)
at
org.apache.pinot.spi.utils.ArrayCopyUtils.copyToTimestamp(ArrayCopyUtils.java:272)
at
org.apache.pinot.core.operator.transform.function.CastTransformFunction.transformToTimestampValuesSV(CastTransformFunction.java:194)
at
org.apache.pinot.core.operator.transform.function.CastTransformFunction.transformToLongValuesSV(CastTransformFunction.java:182)
at
org.apache.pinot.core.operator.docvalsets.TransformBlockValSet.getLongValuesSV(TransformBlockValSet.java:102)
at
org.apache.pinot.core.common.RowBasedBlockValueFetcher.createFetcher(RowBasedBlockValueFetcher.java:61)
at
org.apache.pinot.core.common.RowBasedBlockValueFetcher.<init>(RowBasedBlockValueFetcher.java:35)
```
- Currently, transform functions handle null values through the
`TransformFunction::getNullBitmap` method. However, unlike scalar functions
where null intolerant functions simply skip computation and output `null` for
any input `null` values, transform functions operate on a block basis and
simply use the default null values during computation (with the null bitmap
maintaining context on the null values throughout the transformation chain).
- In the case of a string to timestamp cast, this becomes problematic
because the default null string value (`null`) can't be converted into a valid
timestamp and the transform errors out. This patch fixes the issue by using the
null bitmap to simply skip over the null values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
