Gabriel39 opened a new pull request, #67631:
URL: https://github.com/apache/doris/pull/67631
### What problem does this PR solve?
Issue Number: None
Related PR: #25194
Problem Summary: FileScannerV2 reused `thrift_max_message_size` as the
Parquet footer allocation limit, so valid files with metadata larger than the
100 MiB RPC ceiling were rejected. This PR adds an independent mutable
`parquet_metadata_size_limit` with a 256 MiB default. It preserves the
structural `footer_size <= file_size - 8` validation and keeps the metadata cap
check before allocation and any second read.
### Release note
FileScannerV2 now limits Parquet metadata with the independent
`parquet_metadata_size_limit` configuration, which defaults to 256 MiB.
### Check List (For Author)
- Test: Unit Test
- `NewParquetReaderTest.NativeFooter*` (7 tests passed under ASAN)
- Behavior changed: Yes (valid Parquet metadata between the Thrift RPC
ceiling and the new metadata limit is accepted.)
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
