gstvg commented on issue #18150:
URL: https://github.com/apache/datafusion/issues/18150#issuecomment-4455998823
> Would this potentially require modifying multiple optimizer rules to
ensure they don't get pushed down?
The async_udf being pulled again to the aggr node? Or the new projection
node being pushed further down the tree? (Can this be a problem?) But anyway,
you have a good point, `ProjectionSimplifier` would merge two consecutive
projections (which is how the error I hit would be fixed), and that's only
taking into account core rules, this could require changes in downstream rules,
which is not great. This also wouldn't work for joins where the async_udf can
use inputs from both sides.
> ... clean up this rewrite logic and do it earlier ...
I read the code again and I believe, as you said, that doing it earlier
during physical planning can fix both issues, by creating the physical
expressions using the schema that it's only available after creating the
`AsyncMapper`, and maybe rewriting the logical expressions instead of the
physical ones
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
