bucko909 opened a new pull request, #5837:
URL: https://github.com/apache/hadoop/pull/5837
I had an invalid URL for this property. The NodeManager output:
```
2023-07-13 10:14:58,397 ERROR logaggregation.AppLogAggregatorImpl: Error
occurred while aggregating the log for the application
application_1689169074550_0019
java.lang.NullPointerException
at
org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogWriter.close(AggregatedLogFormat.java:566)
at
org.apache.hadoop.yarn.logaggregation.filecontroller.tfile.LogAggregationTFileController.closeWriter(LogAggregationTFileController.java:99)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.uploadLogsForContainers(AppLogAggregatorImpl.java:395)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.doAppLogAggregation(AppLogAggregatorImpl.java:525)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.run(AppLogAggregatorImpl.java:472)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService$1.run(LogAggregationService.java:295)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
2023-07-13 10:14:58,398 WARN logaggregation.AppLogAggregatorImpl: Log
aggregation did not complete for application application_1689169074550_0019
```
This was caused by the initialise() function failing and not setting
`fsDataOStream`, with the exception being swallowed by a crash in the crash
handler.
After installing a `.jar` built using this patch, I got:
```
2023-07-13 10:15:34,175 ERROR logaggregation.AppLogAggregatorImpl: Error
occurred while aggregating the log for the application
application_1689169074550_0019
org.apache.hadoop.HadoopIllegalArgumentException: Uri without authority:
hdfs:/var/log/yarn/<REMAINDER REMOVED FROM REPORT>
at
org.apache.hadoop.fs.AbstractFileSystem.getUri(AbstractFileSystem.java:330)
at
org.apache.hadoop.fs.AbstractFileSystem.<init>(AbstractFileSystem.java:282)
at org.apache.hadoop.fs.Hdfs.<init>(Hdfs.java:80)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.fs.AbstractFileSystem.newInstance(AbstractFileSystem.java:143)
at
org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:181)
at
org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:266)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:347)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:344)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at
org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:344)
at
org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:471)
at
org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogWriter$1.run(AggregatedLogFormat.java:477)
at
org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogWriter$1.run(AggregatedLogFormat.java:474)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
at
org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogWriter.initialize(AggregatedLogFormat.java:474)
at
org.apache.hadoop.yarn.logaggregation.filecontroller.tfile.LogAggregationTFileController.initializeWriter(LogAggregationTFileController.java:88)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.uploadLogsForContainers(AppLogAggregatorImpl.java:326)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.doAppLogAggregation(AppLogAggregatorImpl.java:525)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl.run(AppLogAggregatorImpl.java:472)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService$1.run(LogAggregationService.java:295)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
```
This is a much more useful error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
