Michael Chen created SPARK-50853:
------------------------------------
Summary: Temp shuffle files not closed
Key: SPARK-50853
URL: https://issues.apache.org/jira/browse/SPARK-50853
Project: Spark
Issue Type: Bug
Components: Shuffle, Spark Core
Affects Versions: 3.4.0
Reporter: Michael Chen
SimpleDownloadWritableChannel implementation of closeAndRead method is not
closing the writable channel. (compared to EncryptedDownloadWritableChannel
implementation which is)
{code:java}
@Override
public ManagedBuffer closeAndRead() throws IOException {
return new FileSegmentManagedBuffer(transportConf, file, 0, file.length());
} {code}
There is nothing explicitly closing the channel, and it's relying on
finalizer/phantom cleanable to close the file descriptor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
