huaheh opened a new issue, #9188:
URL: https://github.com/apache/pinot/issues/9188
In a partial upsert table, we can define an upsert strategy for each column,
for example, OVERWRITE. This leads to the column's value always being
overwritten by the next incoming event. The suggestion is to have control over
merging the existing column value and the field value of the incoming event.
For example, a table has two columns, col1 and col2, both having OVERWRITE
upsert strategy. With incoming event {"f1":"v1", "f2": null}, the expected
transformation is like
```
{
"columnName": "col1",
"transformFunction": "OVERWRITE_IF(f1 != null, f1, col1)"
},
{
"columnName": "col2",
"transformFunction": "OVERWRITE_IF(f2 != null, f2, col2)"
},
```
This way col1 will be updated with v1 while col2 will remain its old value.
This flexibility allows us to ingest some fields of an event and drop the other
fields, and not have to drop the whole event if we merely don't want some
columns to be overwritten.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
