siddharthteotia commented on pull request #8398: URL: https://github.com/apache/pinot/pull/8398#issuecomment-1081386541
> I'd like to request reaching consensus here: > > * Do we agree that using raw values for high cardinality metrics makes sense? If so, can we enable no-dict for those either by default or under a flag? > * Do we agree that [Combine Metadata and Dictionary based plans into single operator #8408](https://github.com/apache/pinot/pull/8408) takes care of issue 1 in @siddharthteotia 's comment? And issues 2 and 3 from the same comment don't apply to this PR. > * We can independently make the heuristic between recommendation engine / auto-tuner and this PR use the same algorithm, but that should be outside the scope of this PR. > > If there are more open items, lets reach consensus on this, and move froward with this PR. > > cc: @siddharthteotia @richardstartin @Jackie-Jiang @mayankshriv my comment was more of a FYI to share our learnings from doing something similar with recommender rule engine. I don't mean to increase scope of this PR. I agree that for high cardinality metrics, no dictionary is a better option. I think outright if we want to create metric columns with no dictionary, regardless of cardinality, then we should probably do that under a flag to keep the default behavior same as today ? I will review @richardstartin PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
