devoopsman45 commented on PR #1881: URL: https://github.com/apache/datafusion-ballista/pull/1881#issuecomment-4755218005
> > SessionContext::remote_with_state only stores the table locally on the client, it is not reaching the scheduler. > > Yes, tables are only kept at the client. If they don't reach scheduler that probably means paths are wrong Thanks. To clarify, the EmptyExec appears in the scheduler's plan before it even reaches the executor. I verified this by mounting the data directory at the exact same absolute path inside the executor container, and the scheduler logs still showed EmptyExec at the leaf. So the file path is reachable by the executor, but the scan operator is already lost at the serialization step on the client side. It seems to be a bug on the _BallistaLogicalExtensionCodec_. I will try to dig deep. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
