[ https://issues.apache.org/jira/browse/HADOOP-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112613#comment-16112613 ]
Steve Loughran commented on HADOOP-13134: ----------------------------------------- WORKAROUND: tell the job committer to ignore failures in cleanup As discussed in [Spark Cloud Integration|https://github.com/apache/spark/blob/master/docs/cloud-integration.md], you can downgrade failures during cleanup to warnings. I recommend this against object stores for a slightly more robust commit, given that directory delete is a more complex/brittle operation.& more prone to failures {code} spark.hadoop.mapreduce.fileoutputcommitter.cleanup-failures.ignored true {code} > WASB's file delete still throwing Blob not found exception > ---------------------------------------------------------- > > Key: HADOOP-13134 > URL: https://issues.apache.org/jira/browse/HADOOP-13134 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure > Affects Versions: 2.7.1 > Reporter: Lin Chan > Assignee: Dushyanth > > WASB is still throwing blob not found exception as shown in the following > stack. Need to catch that and convert to Boolean return code in WASB delete. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org