bharadwaj-pendyala opened a new pull request, #2867: URL: https://github.com/apache/datafusion-ballista/pull/2867
# Which issue does this PR close? Closes #2840. # Rationale for this change andygrove reported in #2840 that a push-staged job whose task can't be prepared fails with "task could not be prepared or was rejected by executor", while the reason only reaches the scheduler log. Pull-staged already puts it in the status. After #2721, this hides the error that names an unrestrictable leaf in a split stage. # What changes are included in this PR? - Return preparation errors from `launch_multi_task` separately from executor rejections. - Merge those errors across executors in `launch_tasks`. `revive_offers` uses the preparation error when there is one, or "task was rejected by executor <id>" when there isn't. - Fix the sentence in the 56.0.0 upgrade note that says push-staged status only has the generic text. The `TaskLauncher` trait is unchanged. If several tasks of one job fail to prepare in the same round, the job reports one of those errors. Test plan: - [x] New `revive_offers_fails_the_job_with_the_preparation_error`: a split `PayloadExec` stage with no resolver. Before the fix the status has the generic text, after it names `PayloadExec`. - [x] New `launch_tasks_prefers_the_preparation_error_to_a_rejection`: the same job is rejected on one executor and fails to prepare on another, and the status keeps the preparation error. - [x] The existing rejection test checks the new rejection text. - [x] `cargo test -p ballista-scheduler --lib` and `dev/rust_lint.sh` pass. # Are there any user-facing changes? Yes. Push-staged job status, and the history server's job end event, now carry the preparation error. A job an executor rejects now fails with "task was rejected by executor <id>", so anything matching the old combined message needs updating. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
