Siddharth Ahuja created YARN-10224:
--------------------------------------
Summary: Exception resulting from failure of a corrupted IFile log
in JHS UI incorrectly logged as WARN with no exception stacktrace provided
Key: YARN-10224
URL: https://issues.apache.org/jira/browse/YARN-10224
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Siddharth Ahuja
While investigating YARN-10207, it was noticed that when an IFile (aggregated
log in HDFS in IndexedFormat) and is being attempted to be rendered inside the
JHS UI, an IOException is (correctly) encountered inside a try block.
The finally clause here ->
https://github.com/apache/hadoop/blob/4af2556b48e01150851c7f273a254a16324ba843/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/LogAggregationIndexedFileController.java#L900
helps cleaning up the socket connection by closing out FSDataInputStream.
However, only a WARN log is presented to the user in the JHS logs in case of
rendering failure and there is no stacktrace logged coming from the exception
here -
https://github.com/apache/hadoop/blob/c24af4b0d6fc32938b076161b5a8c86d38e3e0a1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/IndexedFileAggregatedLogsBlock.java#L136
as the exception is just swallowed up inside the catch{} clause. This prevents
understanding the cause of the rendering failure.
We should log the message as an ERROR along with the exception coming from the
render failure. This is how it works with the TFile rendering failure as
identified in YARN-10207.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
