rdettai opened a new pull request #1072:
URL: https://github.com/apache/arrow-datafusion/pull/1072
# Which issue does this PR close?
This is linked to #1010 #1062
# Rationale for this change
Currently, we don't have a good way to make the `ObjectStoreRegistry`
available in various places of the code. Indeed, to setup its object stores,
the registry needs to have access to the implementation code (which might be in
an other crate) at compile time and needs to have access to the proper
configurations (such as credentials) when initialized. In particular in
Ballista, we need to have the right registry configuration on the Executors
_and_ the Scheduler.
Creating a singleton static object store registry is not ideal (it comes
with with the usual curses that go with static), but solves (at least
temporarily) some of the issues:
- To add a new object store, we now have a single point of configuration,
the `OBJECT_STORES` initialization, that will propagate the configuration to
all the places where the registry is required (local, scheduler, executor)
- `OBJECT_STORES` can be accessed in any place of the code: logical plan or
physical plan deserialization, provider creation...
# What changes are included in this PR?
- removing the registry from the `ExecutionContextState`
- exposing a static `ObjectStoreRegistry` called `OBJECT_STORES`
# Are there any user-facing changes?
- removing the registry from the `ExecutionContextState` changes the config
API but that part was likely not used yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
