ulysses-you opened a new pull request #25333: [SPARK-28597][SS]Spark streaming terminated when close meta data log error URL: https://github.com/apache/spark/pull/25333 ## What changes were proposed in this pull request? Now, HDFSMetaDataLog is the only implement of MetaDataLog. If HDFSMetaDataLog.writeBatchToFile() execute error, then total app will be terminated. The meta data log is very important for spark, we should retry some times when exception, and add a config to control retry times. For example HDFS has a bug that close error if has no enough replicas. More detail [HDFS-11486](https://issues.apache.org/jira/browse/HDFS-11486) ## How was this patch tested?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
