GitHub user shadowmmu added a comment to the discussion: [Performance] Velox Bloom Filter Inefficiency vs. Photon at 1TB Scale
@zhouyuan thanks for your response, in the above example the bloom filter is not triggered but I tried increasing the creation side threshold to 1gb and the runtime filter triggered, however it only filtered only few thousand rows. So then I tried to play with the `spark.gluten.sql.columnar.backend.velox.bloomFilter.maxNumBits` but it failed like can not exceed from default values. and so it said in the below screenshot as well which is from velox official documentation. <img width="1311" height="299" alt="image" src="https://github.com/user-attachments/assets/9ac25d9c-6ea2-4eeb-ab11-9e5acb2a6c65"; /> GitHub link: https://github.com/apache/incubator-gluten/discussions/11554#discussioncomment-15685979 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
