[
https://issues.apache.org/jira/browse/SPARK-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073548#comment-14073548
]
Apache Spark commented on SPARK-2671:
-------------------------------------
User 'sarutak' has created a pull request for this issue:
https://github.com/apache/spark/pull/1580
> BlockObjectWriter should create parent directory when the directory doesn't
> exist
> ---------------------------------------------------------------------------------
>
> Key: SPARK-2671
> URL: https://issues.apache.org/jira/browse/SPARK-2671
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Kousuke Saruta
> Priority: Minor
>
> BlockObjectWriter#open expects parent directory is present.
> {code}
> override def open(): BlockObjectWriter = {
> fos = new FileOutputStream(file, true)
> ts = new TimeTrackingOutputStream(fos)
> channel = fos.getChannel()
> lastValidPosition = initialPosition
> bs = compressStream(new BufferedOutputStream(ts, bufferSize))
> objOut = serializer.newInstance().serializeStream(bs)
> initialized = true
> this
> }
> {code}
> Normally, the parent directory is created by DiskBlockManager#createLocalDirs
> but, just in case, BlockObjectWriter#open should check the existence of the
> directory and create the directory if the directory does not exist.
> I think, recoverable error should be recovered.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
