pan3793 commented on PR #6335: URL: https://github.com/apache/kyuubi/pull/6335#issuecomment-2269277556
general idea: 1. for batch impl v1, `spark-submit` runs immediately on the same kyuubi instance after "create batch" request is accepted, and the existing mechanism ensures that each batch job must be recovered from the same kyuubi instance, plus the above "session folder for extra resources idea", we don't need to track the resource file list, so no extra metadata(and DB schema change) is required, right? 2. for batch impl v2, "create batch" request just inserts a record into DB and waits for all kyuubi instances to pick up, but for batch jobs that have uploaded resource files, we must ensure that the job will be picked up by the kyuubi instance that accepted the "create batch" request. I think we currently have such issues, which can be addressed later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org For additional commands, e-mail: notifications-h...@kyuubi.apache.org
