EricGao888 commented on issue #12394: URL: https://github.com/apache/dolphinscheduler/issues/12394#issuecomment-1281843801
> This issue said if we want to run a .ipynb script via papermill on a remote server, we have to implement CustomEngine. [nteract/papermill#361](https://github.com/nteract/papermill/issues/361) > > In DolphinScheduler, can we exec in pod directly? Such as we start a JupyterLab with custom image in kubelow which have dolphinscheduler worker? Or other connect method? @jieguangzhou Two ways to do so: 1. If there is `Yarn` in remote cluster, you could make papermill run in yarn containers. 2. If the remote cluster is a K8S cluster, you could build papermill, jupyter and related libs into an image and run it in K8S pod. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
