rohangarg commented on PR #13201: URL: https://github.com/apache/druid/pull/13201#issuecomment-1277362342
@paul-rogers : Thanks for the suggestion and explanation! :) Yes, we already do have a configurable limit (default 10k elements) on cardinality for this conversion. If the number of elements in the join table are above that limit, we don't convert the join as a filter and retain it as is. If the value is less than the threshold, then we convert the join to an `InFilter`. The advantage being that with `InFilter` the filter values can potentially use bitmap filtering instead of running the value matching based join filter. The joins in druid currently are always broadcast and we always materialize the build table before planning the physical join execution. In multi-join case, maybe cost estimation (using probe and builds' cardinality) would be useful to decide which join to run first. To force the execution of join or filter, currently you can toggle the configurable limit. As the limit is decreased, more things are run as join themselves without any filter conversions. And as the limit increases, the join to filter conversions increase. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
