[
https://issues.apache.org/jira/browse/OOZIE-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774007#comment-16774007
]
Shubham edited comment on OOZIE-3439 at 2/21/19 11:52 AM:
----------------------------------------------------------
[~kmarton],
Hive1 action log files has different log entries, so pattern is different for
Hive1. (https://issues.apache.org/jira/browse/OOZIE-2112)
Hive1 :
{code:java}
2019-02-20 14:01:36,055 [main] INFO
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application
application_1550671202870_0002
2019-02-20 14:01:46,498 [main] INFO
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager - The current user:
hive, session user: hive
2019-02-20 14:01:46,498 [main] INFO
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager - Current queue name
is default incoming queue name
{code}
But for Hive2, we do not have same log entries for yarn application.
Hive2:
{code:java}
INFO : Status: Running (Executing on YARN cluster with App id
application_1550671202870_0004)
ESC[2K--------------------------------------------------------------------------------
ESC[2KESC[36;1m VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
ESC[22;0mESC[2K--------------------------------------------------------------------------------
ESC[2KMap 1 .......... SUCCEEDED 1 1 0 0 0 0
ESC[2K--------------------------------------------------------------------------------
ESC[2KESC[31;1mVERTICES: 01/01 [==========================>>] 100% ELAPSED
TIME: 5.10 s
ESC[22;0mESC[2K------------------------------------------------------------
{code}
was (Author: shubham.chhabra):
[~kmarton],
Hive1 action log files has different log entries, so pattern is different for
Hive1. (https://issues.apache.org/jira/browse/OOZIE-2112)
Hive1 :
{code}
2019-02-20 14:01:36,055 [main] INFO
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application
application_1550671202870_0002
2019-02-20 14:01:46,498 [main] INFO
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager - The current user:
hive, session user: hive
2019-02-20 14:01:46,498 [main] INFO
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager - Current queue name
is default incoming queue name
{code}
But for Hive2, we do not have same log entries for yarn application.
Hive2:
{code}
INFO : Status: Running (Executing on YARN cluster with App id
application_1550671202870_0004)
ESC[2K--------------------------------------------------------------------------------
ESC[2KESC[36;1m VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
ESC[22;0mESC[2K--------------------------------------------------------------------------------
ESC[2KMap 1 .......... SUCCEEDED 1 1 0 0 0 0
ESC[2K--------------------------------------------------------------------------------
ESC[2KESC[31;1mVERTICES: 01/01 [==========================>>] 100% ELAPSED
TIME: 5.10 s
ESC[22;0mESC[2K------------------------------------------------------------
{code}
> Hive2 action is not parsing application ID for TEZ from log file properly
> -------------------------------------------------------------------------
>
> Key: OOZIE-3439
> URL: https://issues.apache.org/jira/browse/OOZIE-3439
> Project: Oozie
> Issue Type: Bug
> Components: action
> Affects Versions: trunk
> Reporter: Shubham
> Assignee: Shubham
> Priority: Major
> Attachments: OOZIE-3439-001.patch
>
>
> Oozie workflow does not populate ChildJobUrl for Hive2 Action while Hive1 is
> able to find child job ids.
> I looked at the code and found that pattern is not correct for hive2 action
> logs generated in usercache.
> {code:java}
> static final Pattern[] HIVE2_JOB_IDS_PATTERNS = {
> Pattern.compile("Ended Job = (job_\\S*)"),
> Pattern.compile("Submitted application (application[0-9_]*)"),
> Pattern.compile("Running with YARN Application = (application[0-9_]*)")
> }
> {code}
> Adding below pattern should help in getting Hive 2 action Tez application id
> {code:java}
> Pattern.compile("Executing on YARN cluster with App id (application[0-9_]*)"),
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
