nizarhejazi opened a new issue #8374:
URL: https://github.com/apache/pinot/issues/8374
Currently, if:
1. `nullHandlingEnabled` is set to True.
2. Inbuilt DateTime functions that cannot take null (e.g. `FromDateTime`,
`ToDateTime`, etc.) are used as ingestion transforms (in transformConfigs).
3. Input column value is null.
Then: these inbuilt functions throws `java.lang.NullPointerException`
instead of marking the value as null in the null index.
Example:
```
"ingestionConfig": {
"transformConfigs": [
{
"columnName": "updatedAt_timestamp",
"transformFunction": "FromDateTime(updatedAt,
'yyyy-MM-dd''T''HH:mm:ss.SSS''Z''')"
}
]
}
...
"nullHandlingEnabled": True,
```
Note: Cannot use inbuilt functions in Groovy scripts (otherwise the
following exception is thrown: MissingPropertyException: No such property:
DateTimeFormat for class: Script1).
Recommendation:
- Add an annotation for DateTime functions that cannot take null, and just
return null when the function is annotated if used in transformConfigs.
Question: What to do if end user wrote a Groovy script as an ingestion
transform and used a function that cannot handle nulls?
Planning to send a PR to address the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
