ZanderXu opened a new pull request, #4644: URL: https://github.com/apache/hadoop/pull/4644
### Description of PR In our prod environment, we occasionally encounter MaxDirectoryItemsExceededException caused job failure. ``` org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$MaxDirectoryItemsExceededException): The directory item limit of /user/XXX/.sparkStaging is exceeded: limit=1048576 items=1048576 ``` In order to avoid it, we add a metric to sense possible MaxDirectoryItemsExceededException in time. So that we can process it in time to avoid job failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
