tbar4 commented on PR #1100: URL: https://github.com/apache/datafusion-ballista/pull/1100#issuecomment-2480048712
@milenkovicm I am using the deprecated BallistaContext, but just to define config and then passing the context created from the deprecated BallistaContext to the Python DataFusion SessionContext. I have tested this out on a 3 node cluster (AWS r5.2XLarge, 1 scheduler and 2 executors) and work is being routed to both executors. The current SessionContextExt doesn't take any config options as an argument and instead takes the state which contains a config. In a future PR I can make the switch if necessary, but this PR lets people get up an running with Ballista in Python and utilize all the features of DataFusion Python. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
