pantShrey opened a new pull request, #9749: URL: https://github.com/apache/arrow-rs/pull/9749
# Which issue does this PR close? N/A - this is a small API parity fix. # Rationale for this change `StreamReader` and `FileDecoder` both expose `with_skip_validation` but `StreamDecoder` does not, despite having the same internal `skip_validation` field. Motivation: DataFusion's spill infrastructure is being refactored to use `StreamDecoder` for reading spill files via a pluggable backend trait ([apache/datafusion#21215](https://github.com/apache/datafusion/issues/21215)). Since DataFusion controls what it writes it can trust its own IPC output, and needs this method to maintain the same performance characteristic as the current StreamReader-based implementation # What changes are included in this PR? Adds `with_skip_validation` to `StreamDecoder` mirroring the existing implementation on `FileDecoder` # Are these changes tested? The existing StreamDecoder tests cover the decoding path. No new tests added as this method sets an internal flag already tested via `FileDecoder` and `StreamReader`. # Are there any user-facing changes? Yes, `with_skip_validation` is now available on `StreamDecoder`. It is marked unsafe with the same safety requirements as `FileDecoder::with_skip_validation`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
