alamb commented on PR #103: URL: https://github.com/apache/datafusion-site/pull/103#issuecomment-3275500635
> @adriangb @alamb what is still unclear for me, how the dynamic filter gets the min value? is it under the hood by scanning lets say stats, like parquet file footers or histograms? @comphead -- the min value is the minimum value of what has *already been seen* during query execution. * So when the query starts, there is no min value. * As data flows through and the top K operator starts updating the heap, the min value is set * As more data flows through and the heap is refined (new values potentially replace existing values) then the min value can be updated as well The TopK operator heap here: https://github.com/apache/datafusion/blob/ab108a50d75e4e12fb6ebbfac0d0bffa24c265ea/datafusion/physical-plan/src/topk/mod.rs#L119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
