mingmwang commented on issue #2136:
URL:
https://github.com/apache/arrow-datafusion/issues/2136#issuecomment-1089871837
> > 2. register_object_store
>
> > Do you modify the Ballista source code and try to register
s3-objectstore ? You need to register the objectstore to runtime env in
multiple components: Ballista Client, Ballista Scheduler and Ballista Executor.
>
> I think the correct logic is: use the code
"sessionContext.runtime_env().register_object_store("s3",Arc::new(S3FileSystem{}));"
to register the objectstore, Scheduler and Executor to obtain the serialized
s3-objectstore through the network , instead of writing the code for
registering s3-objectstore in both Scheduler and Executor. If you do that, the
parameters of the objectstore also need to be passed over the network.
@mingmwang
Yes, ideally the objectstore just need to be registered once. But currently
since there is no general serialization API for the
objectstore, this is unlike JVM languages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
