[ https://issues.apache.org/jira/browse/SPARK-22788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16291602#comment-16291602 ]
Apache Spark commented on SPARK-22788: -------------------------------------- User 'vanzin' has created a pull request for this issue: https://github.com/apache/spark/pull/19983 > HdfsUtils.getOutputStream uses non-existent Hadoop conf "hdfs.append.support" > ----------------------------------------------------------------------------- > > Key: SPARK-22788 > URL: https://issues.apache.org/jira/browse/SPARK-22788 > Project: Spark > Issue Type: Bug > Components: DStreams > Affects Versions: 1.6.0 > Reporter: Marcelo Vanzin > Priority: Trivial > > Code: > {noformat} > if (conf.getBoolean("hdfs.append.support", false) || > dfs.isInstanceOf[RawLocalFileSystem]) { > dfs.append(dfsPath) > } else { > throw new IllegalStateException("File exists and there is no append > support!") > } > {noformat} > This makes the exception to be thrown if you enable > {{writeAheadLog.closeFileAfterWrite}} with HDFS. > The correct config, from {{DFSConfigKeys}}, is {{dfs.support.append}}, and > the default value is true. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org