rdettai commented on pull request #1072:
URL: https://github.com/apache/arrow-datafusion/pull/1072#issuecomment-939727247
> I don't see a strong reason why we want to avoid referencing execution
context during logical plan deserilization?
The context contains a lot of different configurations, part of which will
will be copied into the logical plan, part of which won't. So it seems to me
that it will be kind of hard to figure out which configurations need to be
consistently set across the different execution contexts across the cluster
(either through the boot time config or through serialization along the query),
and which are only needed on the node where the plan is created. I guess that
in that case, the context should be structured into multiple tiers:
- local (ok if configured on client instance only)
- user defined variables
- to be serialized along the logical plan
- ex: batch size
- static (boot time, needs to be configured on all nodes separately)
- ex: object_store_registry, optimizers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
