Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21684#discussion_r200773678
--- Diff:
core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionClient.scala ---
@@ -233,30 +233,43 @@ private[spark] class RestSubmissionClient(master:
String) extends Logging {
private[rest] def readResponse(connection: HttpURLConnection):
SubmitRestProtocolResponse = {
import scala.concurrent.ExecutionContext.Implicits.global
val responseFuture = Future {
- val dataStream =
- if (connection.getResponseCode == HttpServletResponse.SC_OK) {
- connection.getInputStream
- } else {
- connection.getErrorStream
+ val responseCode = connection.getResponseCode
+
+ if (responseCode != HttpServletResponse.SC_OK) {
+ val errString =
Some(Source.fromInputStream(connection.getErrorStream())
+ .getLines().mkString("\n"))
+ logError(s"Server responded with error:\n${errString}")
+ if (responseCode == HttpServletResponse.SC_INTERNAL_SERVER_ERROR
&& connection.getRequestMethod == "GET") {
--- End diff --
OK, so this ensures that the behavior actually stays the same. OK, I think
that's a little bit safer, even if later it'd be fine to change the behavior in
Spark 3 to simply be more consistent with itself.
Is it necessary to restrict this case to GET and a 500 response? In
particular is it necessary to behave differently for POST? that almost sounds
like the current behavior doesn't make sense then if it differs.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
