[
https://issues.apache.org/jira/browse/SPARK-57958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-57958.
----------------------------------
Fix Version/s: 4.1.3
3.5.9
4.0.4
4.2.1
4.3.0
Resolution: Fixed
Issue resolved by pull request 57036
[https://github.com/apache/spark/pull/57036]
> Close the ORC Reader in OrcSourceSuite to fix leaked file streams
> -----------------------------------------------------------------
>
> Key: SPARK-57958
> URL: https://issues.apache.org/jira/browse/SPARK-57958
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.3, 3.5.9, 4.0.4, 4.2.1, 4.3.0
>
>
> OrcSuite.testBloomFilterCreation and testSelectiveDictionaryEncoding create
> an ORC Reader via OrcFile.createReader(...) but only close the derived
> RecordReader in the finally block, never the Reader itself. The Reader holds
> the file stream opened in ReaderImpl.extractFileTail, so it leaks.
> SharedSparkSession.afterEach asserts no open streams
> (DebugFilesystem.assertNoOpenStreams) and, when the leaked stream is not
> closed within the 10s window, aborts the whole suite: 'There are 1 possibly
> leaked file streams'. This flakes OrcSourceV1Suite/OrcSourceV2Suite, e.g. on
> build_java21 (branch-4.0). Fix: also close the Reader in the finally block.
> Test-only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
