[
https://issues.apache.org/jira/browse/HIVE-27126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18081201#comment-18081201
]
László Bodor commented on HIVE-27126:
-------------------------------------
Reconsidering: so this one can be implemented independently from HIVE-25666, as
this is tightly coupled with Yarn. One consideration is whether HS2 should be
able to query this from Yarn ResourceManager, or the Tez client can get back
such information from Tez AMs. I believe this is clearly about Yarn resources,
which might be available from Yarn RM, so HS2 should be able to query that kind
of information, and we should not pollute the DagClient with that kind of info.
We can also always assume that HS2 is able to communicate with Yarn RM any
time, as it submits Tez applications by using it, e.g. here:
https://github.com/apache/hive/blob/1516fb91e8a622b4655b0e07a37fb449634c1637/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java#L695.
Even if this is inside the TezClient, HS2 should be able to utilize the same
code without the TezClient.
Regarding real-time beeline progress, I believe the abandoned PR
(https://github.com/apache/hive/pull/2812) of HIVE-25666 could be a good base.
Regarding what exactly can be rendered in beeline, we can re-iterate about, cc:
[~tarak271].
> Display Yarn resources availability in real time along with Tez job progress
> -----------------------------------------------------------------------------
>
> Key: HIVE-27126
> URL: https://issues.apache.org/jira/browse/HIVE-27126
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
> Reporter: Taraka Rama Rao Lethavadla
> Priority: Major
>
> Right we don't know if the queue used in Hive query execution is having
> enough resources or not. We are already displaying tez job details like
> number of task per each vertex and how are they progressing.
> If the resources available are not good enough to execute the query in
> parallel or query that use to take shorter time is taking time, then we can
> use this new detail to understand that queue is busy and resources are not
> enough
>
>
>
> {noformat}
> ----------------------------------------------------------------------------------------------
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
> FAILED KILLED
> ----------------------------------------------------------------------------------------------
> Map 1 container RUNNING 230 0 10 230
> 0 1
> Reducer 2 container INITED 1 0 0 1
> 0 0
> ----------------------------------------------------------------------------------------------
> VERTICES: 00/02 [>>--------------------------] 0% ELAPSED TIME: 58.33 s
> ----------------------------------------------------------------------------------------------
> {noformat}
> So it would be great to display current resource availability in that
> specific queue along with above info in real time would help understand why a
> query can be slow
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
