[
https://issues.apache.org/jira/browse/HIVE-16601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206080#comment-16206080
]
Sahil Takiar commented on HIVE-16601:
-------------------------------------
Few updates. The screen shots are a bit outdated, it now looks like:
* The Spark App Name will now be "Hive on Spark (sessionId = \[my-session-id\])"
** Note that users can override the Spark App Name by setting
{{spark.app.name}} in their session config; in this case the "(sessionId =
\[my-session-id\])" will be appended to the Spark App Name
** The goal here is to help users map Hive on Spark applications to the session
that launched them
* The Spark Job List will now list all jobs in a Spark application via the
following format:
** "Job Id (Job Group) = \[job-id-number\] (\[Hive-query-id\])" where the Job
Id is some monotonically increasing integer and the Hive-query-id is the query
id determined by Hive
*** The Job Group should group Spark jobs from the same Hive query together
** "Description = \[query-being-run\]\[stage-id\]" where the Description is a
brief description of what the Job is doing, the query-being-run is a shortened
version of the query being run, the stage-id is the Hive stage id
*** The description gives more info as to what the Job is actually doing
*** A custom description on the query can be provided by the user via the
config {{hive.query.name}} - in which case the stage-id will automatically be
appended to the end of the user specified query name
> Display Session Id, Query Name / Id, and Dag Id in Spark UI
> -----------------------------------------------------------
>
> Key: HIVE-16601
> URL: https://issues.apache.org/jira/browse/HIVE-16601
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Attachments: HIVE-16601.1.patch, HIVE-16601.2.patch,
> HIVE-16601.3.patch, HIVE-16601.4.patch, HIVE-16601.5.patch, Spark UI
> Applications List.png, Spark UI Jobs List.png
>
>
> We should display the session id for each HoS Application Launched, and the
> Query Name / Id and Dag Id for each Spark job launched. Hive-on-MR does
> something similar via the {{mapred.job.name}} parameter. The query name is
> displayed in the Job Name of the MR app.
> The changes here should also allow us to leverage the config
> {{hive.query.name}} for HoS.
> This should help with debuggability of HoS applications. The Hive-on-Tez UI
> does something similar.
> Related issues for Hive-on-Tez: HIVE-12357, HIVE-12523
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
