[jira] [Commented] (YARN-2146) Yarn logs aggregation error
[ https://issues.apache.org/jira/browse/YARN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048910#comment-14048910 ] Mit Desai commented on YARN-2146: - I looked at it. The problem is due to the corner case in the fix. I will file another JIRA to track the issue. Thanks [~airbots] Yarn logs aggregation error --- Key: YARN-2146 URL: https://issues.apache.org/jira/browse/YARN-2146 Project: Hadoop YARN Issue Type: Bug Reporter: Chen He when I run yarn logs -applicationId application_xxx /tmp/application_xxx. It creates file, also shows part of logs on the terminal screen, and reports following error: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:430) at java.lang.Long.parseLong(Long.java:483) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readAContainerLogsForALogType(AggregatedLogFormat.java:566) at org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:139) at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:137) at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:199 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2146) Yarn logs aggregation error
[ https://issues.apache.org/jira/browse/YARN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030812#comment-14030812 ] Xuan Gong commented on YARN-2146: - [~airbots] Hey, Chen. Have you figured out why this happens ? I am very curious. Yarn logs aggregation error --- Key: YARN-2146 URL: https://issues.apache.org/jira/browse/YARN-2146 Project: Hadoop YARN Issue Type: Bug Reporter: Chen He when I run yarn logs -applicationId application_xxx /tmp/application_xxx. It creates file, also shows part of logs on the terminal screen, and reports following error: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:430) at java.lang.Long.parseLong(Long.java:483) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readAContainerLogsForALogType(AggregatedLogFormat.java:566) at org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:139) at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:137) at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:199 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2146) Yarn logs aggregation error
[ https://issues.apache.org/jira/browse/YARN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031120#comment-14031120 ] Chen He commented on YARN-2146: --- I think it is because of mismatching during log parsing. I found this problem when I was running a Pig on Tez job running on Hadoop-2.4. Yarn logs aggregation error --- Key: YARN-2146 URL: https://issues.apache.org/jira/browse/YARN-2146 Project: Hadoop YARN Issue Type: Bug Reporter: Chen He when I run yarn logs -applicationId application_xxx /tmp/application_xxx. It creates file, also shows part of logs on the terminal screen, and reports following error: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:430) at java.lang.Long.parseLong(Long.java:483) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readAContainerLogsForALogType(AggregatedLogFormat.java:566) at org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:139) at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:137) at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:199 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2146) Yarn logs aggregation error
[ https://issues.apache.org/jira/browse/YARN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031162#comment-14031162 ] Xuan Gong commented on YARN-2146: - bq. because of mismatching during log parsing When we aggregates the logs into HDFS, we write the file_name and size of the files before we write the log contents. When it tries to read back size of the log, but somehow the mismatching happens. That cause the exception. Not sure why this can happen. Yarn logs aggregation error --- Key: YARN-2146 URL: https://issues.apache.org/jira/browse/YARN-2146 Project: Hadoop YARN Issue Type: Bug Reporter: Chen He when I run yarn logs -applicationId application_xxx /tmp/application_xxx. It creates file, also shows part of logs on the terminal screen, and reports following error: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:430) at java.lang.Long.parseLong(Long.java:483) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readAContainerLogsForALogType(AggregatedLogFormat.java:566) at org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:139) at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:137) at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:199 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2146) Yarn logs aggregation error
[ https://issues.apache.org/jira/browse/YARN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031167#comment-14031167 ] Chen He commented on YARN-2146: --- If you take a look of the log aggregation, you may get some hint there. What if the size of file is not right? Yarn logs aggregation error --- Key: YARN-2146 URL: https://issues.apache.org/jira/browse/YARN-2146 Project: Hadoop YARN Issue Type: Bug Reporter: Chen He when I run yarn logs -applicationId application_xxx /tmp/application_xxx. It creates file, also shows part of logs on the terminal screen, and reports following error: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:430) at java.lang.Long.parseLong(Long.java:483) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readAContainerLogsForALogType(AggregatedLogFormat.java:566) at org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:139) at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:137) at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:199 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2146) Yarn logs aggregation error
[ https://issues.apache.org/jira/browse/YARN-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031172#comment-14031172 ] Chen He commented on YARN-2146: --- It is the same problem in YARN-1670. Yarn logs aggregation error --- Key: YARN-2146 URL: https://issues.apache.org/jira/browse/YARN-2146 Project: Hadoop YARN Issue Type: Bug Reporter: Chen He when I run yarn logs -applicationId application_xxx /tmp/application_xxx. It creates file, also shows part of logs on the terminal screen, and reports following error: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:430) at java.lang.Long.parseLong(Long.java:483) at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.readAContainerLogsForALogType(AggregatedLogFormat.java:566) at org.apache.hadoop.yarn.logaggregation.LogCLIHelpers.dumpAllContainersLogs(LogCLIHelpers.java:139) at org.apache.hadoop.yarn.client.cli.LogsCLI.run(LogsCLI.java:137) at org.apache.hadoop.yarn.client.cli.LogsCLI.main(LogsCLI.java:199 -- This message was sent by Atlassian JIRA (v6.2#6252)
