Github user rekhajoshm commented on a diff in the pull request:
https://github.com/apache/spark/pull/21684#discussion_r200784938
--- Diff:
core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionClient.scala ---
@@ -233,30 +233,43 @@ private[spark] class RestSubmissionClient(master:
String) extends Logging {
private[rest] def readResponse(connection: HttpURLConnection):
SubmitRestProtocolResponse = {
import scala.concurrent.ExecutionContext.Implicits.global
val responseFuture = Future {
- val dataStream =
- if (connection.getResponseCode == HttpServletResponse.SC_OK) {
- connection.getInputStream
- } else {
- connection.getErrorStream
+ val responseCode = connection.getResponseCode
+
+ if (responseCode != HttpServletResponse.SC_OK) {
+ val errString =
Some(Source.fromInputStream(connection.getErrorStream())
+ .getLines().mkString("\n"))
+ logError(s"Server responded with error:\n${errString}")
+ if (responseCode == HttpServletResponse.SC_INTERNAL_SERVER_ERROR
&& connection.getRequestMethod == "GET") {
--- End diff --
Thanks @srowen for the reviews. Fixed. As our intention is to also comply
with existing tests, updated to verify with content_type.
The reason is we have two tests in "client handles faulty server" both of
which return SC_INTERNAL_SERVER_ERROR but one is expected by tests to return
ErrorMessage while other needs to be an exception.I found that strange. Anyhow,
debugged and found one(KillSubmissionResponse) returns a valid json and
other(StatusSubmissionResponse) is an html response.This is due to the internal
behavior of KillRequestServlet and StatusRequestServlet.
This PR keeps tests behavior consistent, while addressing the objective of
SPARK-24470
```
KillSubmissionResponse for KillSubmissionResponse is json/500
{
"action" : "ErrorResponse",
"message" : "Internal server error:
org.apache.spark.deploy.rest.SubmitRestProtocolException: Validation of message
KillSubmissionResponse
failed!\n\torg.apache.spark.deploy.rest.SubmitRestProtocolMessage.validate(SubmitRestProtocolMessage.scala:70)\n\torg.apache.spark.deploy.rest.RestServlet.validateResponse(RestSubmissionServer.scala:196)\n\torg.apache.spark.deploy.rest.RestServlet.sendResponse(RestSubmissionServer.scala:134)\n\torg.apache.spark.deploy.rest.KillRequestServlet.doPost(RestSubmissionServer.scala:223)\n\tjavax.servlet.http.HttpServlet.service(HttpServlet.java:707)\n\tjavax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\torg.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)\n\torg.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:584)\n\torg.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\torg.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\torg.eclipse.jetty.server
.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\torg.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\torg.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\torg.eclipse.jetty.server.Server.handle(Server.java:534)\n\torg.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\torg.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\torg.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)\n\torg.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)\n\torg.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\torg.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\torg.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\torg.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.ja
va:136)\n\torg.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\torg.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tjava.lang.Thread.run(Thread.java:748)",
"serverSparkVersion" : "2.4.0-SNAPSHOT"
}
```
```
StatusSubmissionResponse from /submissions/status/anything is html/500
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /v1/submissions/status/anything. Reason:
<pre> java.lang.ArithmeticException: / by zero</pre></p>
<hr /><a href="http://eclipse.org/jetty";>Powered by Jetty://
9.3.20.v20170531</a><hr/>
</body>
</html>
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
