lasdf1234 commented on code in PR #13373: URL: https://github.com/apache/gravitino/pull/13373#discussion_r4070094883
########## docs/manage-jobs-in-gravitino.md: ########## @@ -276,19 +277,35 @@ curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \ "http://localhost:8090/api/metalakes/example/jobs/runs/{job_id}?includeOutput=true&outputMaxLines=50&outputMaxBytes=8192"; ``` +:::caution +When multiple Gravitino servers share the same metadata store, `gravitino.job.stagingDir` must be on +storage shared by all servers (for example an NFS mount) for the local job executor to return a +job's output from any server. The servers may mount it at different paths. Otherwise, only the +server that ran a job can return its output, and the other servers return empty output rather than +an error. +::: + +The local job executor finds a job's output through a small index file it writes to +`<gravitino.job.stagingDir>/.job-output-index` when the job is submitted: + +- Jobs submitted before Gravitino 2.0.0, or during a rolling upgrade by a server that isn't upgraded Review Comment: Rolling-upgrade limitation is documented here — helpful. LGTM on making the shared-staging-dir requirement prominent; that's the main operational prerequisite for this fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
