yupeng9 commented on pull request #8273:
URL: https://github.com/apache/pinot/pull/8273#issuecomment-1059590693
> Yea I like that. How does something like this look? @yupeng9
>
> ```
> prefixesToRename: [
> {
> "prefixName": "after."
> "prefixReplace": ""
> },
> ...
> ]
> ```
>
> In terms of avoiding collisions with same column names, perhaps we can add
a validation check when the table config is added? I believe there's similar
validation checks in place currently for comparing column names in the configs
against the schema cc @icefury71
>
> Also since the general idea is to allow for renaming columns from the
source data upon ingestion, we could also end up expanding on this via some
different approaches. One approach could be to specify the exact full column
names we want to rename, and specify what we want to rename it to, like how we
have that capability in transform configs. Another approach would be to apply
the config in batch, so like batch removing/renaming common prefixes, or batch
converting snake_case to camelCase, etc. so that it's not tedious to specify
everything like in the first approach. For this PR, I was thinking of just
starting with `prefixRename` but definitely appreciate any thoughts.
I think it could be even simpler:
```
prefixesToRename: { "after.": "" },
```
Yes, validation works, since this is at the schema level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
