abstractdog opened a new pull request, #6955: URL: https://github.com/apache/hive/pull/6955
### What changes were proposed in this pull request? This is a follow-up to HIVE-30059 (#6793). `ParquetColumnVectorProducer.createReadPipeline` no longer falls back to the normal reader when the footer read fails because the thread was interrupted. In that case it rethrows the `IOException`. A failed footer read counts as an interrupt when either: - the current thread's interrupt flag is set, or - the cause chain contains an `InterruptedException`, a `ClosedByInterruptException`, or an `InterruptedIOException`. `SocketTimeoutException` is excluded: it extends `InterruptedIOException` but is a timeout, so it still falls back. When the read is interrupted: - The original exception is rethrown unwrapped. The interrupt stays the root cause, so the limit-reached bailout in `HiveInputFormat.getRecordReader` (HIVE-25751) still returns a `NullRowsRecordReader`. - The thread's interrupt flag is set again, in case a wrapper cleared it. - One INFO line is logged instead of the WARN with a stack trace. Any other footer failure falls back as before. `createReadPipeline` now declares `throws IOException`, which `ColumnVectorProducer` already did. ### Why are the changes needed? When the native pipeline declines a split, `createReadPipeline` returns `null` and `LlapInputFormat.fallback()` serves the split with the stock `VectorizedParquetRecordReader`. HIVE-30059 sends every footer-read `IOException` down that path (design doc §6.1/§7: "Footer read throws IOException → producer logs at WARN and returns null"). Interrupts were caught too. An interrupt doesn't mean the native pipeline can't serve the split. It means the fragment is being killed or the daemon is shutting down. Falling back starts the same footer read again on the stock reader, on a thread that should be unwinding. It also logs a WARN saying the split lost its vectored reads, which hides the real cause. ### Does this PR introduce _any_ user-facing change? No. An interrupted fragment now stops on the native path instead of trying the read again on the stock reader. Results are unchanged. ### How was this patch tested? New unit tests in `TestParquetEncodedDataReader`. They use a local `FileSystem` whose `getFileStatus` throws a chosen exception, so the footer read fails before any data is read: - `testInterruptedFooterReadIsNotAFallback`: an `InterruptedIOException`, a `ClosedByInterruptException`, and an `IOExceptiodException` are each rethrown as the same object, with the interrupt flag set. - `testFailedFooterReadOnAnInterruptedThreadIsNotAFallback`: a plain `IOException` on an interrupted thread is rethrown. - `testFailedFooterReadFallsBack` (control): a plain `IOException` and a `SocketTimeoutException` still fall back (return `n Without the fix, the first two fail with "expected java.io.IOException to be thrown, but nothing was thrown", and the contro The related llap-server tests pass, 82 in total: ``` mvn -pl llap-server test -Dtest=TestParquetEncodedDataReader,TestParquetCachedPageReadStore,TestLowLevelCacheImpl,TestLlapCacheAwareFsInterop,TestLlapRecordReader,TestVectoredReadBuffers,TestParquetNativeIOCacheDisabled,TestOrcEncodedDataReaderIOCacheDisabled ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
