erenavsarogullari commented on a change in pull request #28208:
URL: https://github.com/apache/spark/pull/28208#discussion_r427048060
##########
File path:
sql/core/src/main/scala/org/apache/spark/status/api/v1/sql/SqlResource.scala
##########
@@ -21,46 +21,55 @@ import java.util.Date
import javax.ws.rs._
import javax.ws.rs.core.MediaType
+import scala.util.{Failure, Success, Try}
+
import org.apache.spark.JobExecutionStatus
-import org.apache.spark.sql.execution.ui.{SQLAppStatusStore,
SQLExecutionUIData, SQLPlanMetric}
+import org.apache.spark.sql.execution.ui.{SparkPlanGraph,
SparkPlanGraphCluster, SparkPlanGraphNode, SQLAppStatusStore,
SQLExecutionUIData}
import org.apache.spark.status.api.v1.{BaseAppResource, NotFoundException}
@Produces(Array(MediaType.APPLICATION_JSON))
private[v1] class SqlResource extends BaseAppResource {
+ val WHOLE_STAGE_CODEGEN = "WholeStageCodegen"
+
@GET
def sqlList(
- @DefaultValue("false") @QueryParam("details") details: Boolean,
+ @DefaultValue("true") @QueryParam("details") details: Boolean,
+ @DefaultValue("true") @QueryParam("planDescription") planDescription:
Boolean,
Review comment:
@gengliangwang Thanks for the review.
Please find my comments as follows:
**1-** `planDescription` exposes Physical Plan which covers dataset
`column-names`. Column Names can be thought as customer sensitive data so with
this option, end-users can disable in the light of their use-cases when they
still access metrics.
**2-** For complex queries, `planDescription` can be big string and create
network overhead. In this case, it can be disabled where it is not required and
metrics are required(e.g: time-series monitoring - metrics need to be persisted
but Physical Plan does not) (if makes sense)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
