Hi all, We discussed in one topic (I do not remember where, but it doesn't matter here) about the future of query evaluation in Cloudberry. And one of the ideas (from Lirong I think) was about implementing Alibaba approach with working with runtime query evaluation https://vldb.org/pvldb/vol16/p3636-wu.pdf
This idea captured me since then. What do you think if I try to implement Anser for Cloudberry? There are several solutions in an article - gathering stat, creating bloomfilters, step-by-step sessions creation for slice processing, switch from broadcast to redistribute and so on. In an overall all ideas are spilling around improving runtime behaviour - gather statistics and use it to improve query performance or save computing resources (mainly memory). I'd want to start with the quite simple functionality - working with bloom filters to reduce the number of processed rows. Create them while reading one side of hash join, propagate to the second side and filter out rows before we spent CPU for creating hash table and perform hash probes. We could check how Anser could be implemented in Cloudberry and what is the overall benefit, what are the overhead costs and check if the whole approach is applicable. WBW, Leonid
