comphead commented on issue #11555: URL: https://github.com/apache/datafusion/issues/11555#issuecomment-2286559186
well the problem is AntiJoin needs to wait for the very last right batch to read for the respective left row. I tried couple of options how to identify the very last right batch, ``` self.buffered_data.scanning_offset == 0 or self.buffered_data.scanning_finished() ``` But each of them has its own false positives or false negatives. Perhaps we need a separate function or index to calculate the very last batch. @korowa do you have any other ideas on that, as you contributed a lot to SMJ, appreciate if you can help -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
