Hyukjin Kwon created SPARK-57958:
------------------------------------
Summary: Close the ORC Reader in OrcSourceSuite to fix leaked file
streams
Key: SPARK-57958
URL: https://issues.apache.org/jira/browse/SPARK-57958
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.3.0
Reporter: Hyukjin Kwon
OrcSuite.testBloomFilterCreation and testSelectiveDictionaryEncoding create an
ORC Reader via OrcFile.createReader(...) but only close the derived
RecordReader in the finally block, never the Reader itself. The Reader holds
the file stream opened in ReaderImpl.extractFileTail, so it leaks.
SharedSparkSession.afterEach asserts no open streams
(DebugFilesystem.assertNoOpenStreams) and, when the leaked stream is not closed
within the 10s window, aborts the whole suite: 'There are 1 possibly leaked
file streams'. This flakes OrcSourceV1Suite/OrcSourceV2Suite, e.g. on
build_java21 (branch-4.0). Fix: also close the Reader in the finally block.
Test-only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
