berkaysynnada commented on issue #7611: URL: https://github.com/apache/arrow-datafusion/issues/7611#issuecomment-1730940146
> @Dandandan @metesynnada @berkaysynnada Do you guys have any thoughts on this? Thanks for the explanation :) @metesynnada may give you more accurate answers since has more experience with joins., but let me give you my thoughts. **"If the join on field is empty, the producer does not put Literal(True) in the field, it will just produce None"** Even if they are practically the same thing, I believe it will not be very correct if we give a different meaning to something that is None. Does producing as None cause an error? **"The reason this issue should be labeled as bug is because the current use of post_join_filter is incorrect."** AFAIK there is no distinction between post and pre-filters in datafusion joins, all non-equi filter predicates are collected in one place. Wouldn't it be misleading to make such a distinction in Substrait? Likewise, on and filter are different fields, and collecting them under a single roof can be confusing when producing and consuming. I have not encountered a such combination. From your first explanation, I can understand why you need this but I'm unsure of the best solution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
