mingmwang edited a comment on pull request #2024:
URL:
https://github.com/apache/arrow-datafusion/pull/2024#issuecomment-1076067762
Hi, @thinkharderdev
I think this PR doesn't resolve the issue. One of the key problem is, users
register the UDF/UDFS with SessionContext.register_udf(), but how does the
SessionContexts (ExecutionContext) in all the Executors know the registered
UDF? The information is not propagate to executor side.
For DataFusion, SessionContext.register_udf() is not a problem. But for
Baliista, there are three different kind of SessionContext:
1. the context created in Ballista client and uses the BallistaQueryPlanner
to send logical plans to Scheduler, see the method
create_df_ctx_with_ballista_query_planner()
3. the context created in Scheduler
4. the context created in Executor
I'm going to remove the context created in Executor because I think it does
not make sense to have Executor hold a global SessionContext.
Hi, @yjshen @alamb
Please share your thoughts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
