kishoreg commented on issue #5268:
URL:
https://github.com/apache/incubator-pinot/issues/5268#issuecomment-618723467
Great discussion so far and valid points. My take is filtering should not
even be part of schema definition. This is more of an ingestion config which is
part of the table config. should we move it to table config?
The challenges mentioned by @npawar such as pre-filter vs post filter etc
still remain. I agree with supporting filtering across multiple columns.
Today source field name extractor is looking at the schema but its possible
that we look at the filter spec as well to compute the list of fields to
extract from source.
Another option is to add a boolean to fieldSpec isTransient=true which
means we just extract it from source but not store in Pinot.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
