[GitHub] spark pull request #23068: [SPARK-26098][WebUI] Show associated SQL query in...

[gengliangwang]

Github user gengliangwang commented on a diff in the pull request:

https://github.com/apache/spark/pull/23068#discussion_r234402149
  
--- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala 
---
@@ -56,6 +56,11 @@ private[spark] class AppStatusStore(
 store.read(classOf[JobDataWrapper], jobId).info
   }
 
+  def jobWithAssociatedSql(jobId: Int): (v1.JobData, Option[Long]) = {
--- End diff --

Return JobData and sqlExcuctionId within one look up.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23068: [SPARK-26098][WebUI] Show associated SQL query in...

[gengliangwang]

Github user gengliangwang commented on a diff in the pull request:

https://github.com/apache/spark/pull/23068#discussion_r234402077
  
--- Diff: 
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -70,6 +70,8 @@ private[spark] class AppStatusListener(
   private val liveTasks = new HashMap[Long, LiveTask]()
   private val liveRDDs = new HashMap[Int, LiveRDD]()
   private val pools = new HashMap[String, SchedulerPool]()
+
+  private val SQL_EXECUTION_ID_KEY = "spark.sql.execution.id"
--- End diff --

Here we can't access `SQLExecution.EXECUTION_ID_KEY`. So use a variable for 
the key string.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #23068: [SPARK-26098][WebUI] Show associated SQL query in...

[gengliangwang]

GitHub user gengliangwang opened a pull request:

https://github.com/apache/spark/pull/23068

[SPARK-26098][WebUI] Show associated SQL query in Job page

## What changes were proposed in this pull request?

For jobs associated to SQL queries, it would be easier to understand the 
context to showing the SQL query in Job detail page.
Before code change:


![image](https://user-images.githubusercontent.com/1097932/48659359-96baa180-ea8a-11e8-8419-a0a87c3f30fc.png)

After code change:

![image](https://user-images.githubusercontent.com/1097932/48659390-26f8e680-ea8b-11e8-8fdd-3b58909ea364.png)


We can see the whole context when in the associated SQL detail page:
![Uploading image.png…]()

For Jobs don't have associated SQL query, the text won't be shown.

## How was this patch tested?

Manual test


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gengliangwang/spark addSQLID

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/23068.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #23068


commit e7c2ebbda949918034cb9cb92ac6ef30af17d943
Author: Gengliang Wang 
Date:   2018-11-17T05:42:26Z

add sql id




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org