chuxiao created KYLIN-4163:
------------------------------
Summary: CreateFlatHiveTableStep has not yarn app url when hive
job running
Key: KYLIN-4163
URL: https://issues.apache.org/jira/browse/KYLIN-4163
Project: Kylin
Issue Type: Improvement
Components: Job Engine
Affects Versions: v3.0.0-alpha
Reporter: chuxiao
Attachments: flathivetablerunning图.jpg
CreateFlatHiveTableStep has yarn app url on the monitor web page only when job
finished, but SparkExecutable has yarn app url when job running.
this is because of SparkExecutable`s logger has logger listener:
{code:java}
final PatternedLogger patternedLogger = new PatternedLogger(logger, new
PatternedLogger.ILogListener() {
@Override
public void onLogEvent(String infoKey, Map<String, String> info) {
// only care three properties here
if (ExecutableConstants.SPARK_JOB_ID.equals(infoKey)
|| ExecutableConstants.YARN_APP_ID.equals(infoKey)
|| ExecutableConstants.YARN_APP_URL.equals(infoKey)) {
getManager().addJobInfo(getId(), info);
}
}
});{code}
sometimes creating flat hive table hangs, so user wants to have yarn app url
when hive job running like attachment.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
