LuciferYang opened a new issue, #9575: URL: https://github.com/apache/paimon/issues/9575
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master, `2788fe596` (2.1-SNAPSHOT). ### Compute Engine Flink and Spark, on any Parquet read where `createReader` fails after the file has been opened. ### Minimal reproduce step Read a Parquet column whose stored type does not match the read type, so that `VectorizedParquetRecordReader`'s constructor rejects it: ``` java.io.IOException: Schema evolution not supported. at org.apache.paimon.format.parquet.reader.VectorizedParquetRecordReader.checkColumn(...) at org.apache.paimon.format.parquet.reader.VectorizedParquetRecordReader.<init>(...) at org.apache.paimon.format.parquet.ParquetReaderFactory.createReader(...) ``` `ParquetReaderFactory.createReader` closes what it opened if reading the footer or constructing the `ParquetFileReader` throws, and both of those paths already have a `catch (Throwable)` that closes and rethrows. The stretch after that does not: building the shredding read plan, resolving the requested schema, checking the batch size, allocating the writable vectors, and constructing `VectorizedParquetRecordReader`. At that point the `ParquetFileReader` is only a local variable, so when one of those throws the caller has no way to close it and the open stream is lost. Other ways into the same stretch, none of which need a damaged file: a case-insensitive read of two columns differing only in case throws `Found duplicate field(s)` from the requested-schema build, and a required column missing from the file throws from the same constructor. ### What doesn't meet your expectations? The two earlier stages of the same method already clean up after themselves, so the third should too. It matters more than a one-off leak because `DataFileRecordReader` treats an `IOException` or `RuntimeException` from `createReader` as a corrupt file when `scan.ignore-corrupt-files` is on: it logs a WARN, returns null and the scan continues, so a scan over many mismatched files loses one descriptor each and nothing says so. ### Anything else? No behaviour change for a successful read: the reader closes the stream as before. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
