Gabriel39 opened a new pull request, #67922: URL: https://github.com/apache/doris/pull/67922
### What problem does this PR solve? Backport #67906 to branch-4.1. TopN two-phase materialization can fetch selected Parquet and ORC file rows through FileScannerV2. The second phase follows the scanner rollout policy, avoids whole-chunk Parquet prefetch for sparse row selections, and preserves partition/generated/synthesized column categories and Iceberg file metadata. Compatibility adjustments for branch-4.1: - Preserve the existing Lance dataset-level uint64 row-ID fetch path, physical split scheduling, condition-cache state, and Variant projections. - Apply name-based column classification to the existing IcebergScanNode and keep relation-snapshot schema categories separate from physical column positions. - Use the existing Iceberg v3 row-lineage regression suite for eager/lazy comparisons; the master-only `_file`/`_pos` feature and connector framework are not prerequisites for this backport. ### Release note Support row-id fetch for Parquet and ORC in FileScannerV2. ### Check List (For Author) - FE: `FileQueryScanNodeTest` passed (14 tests, zero failures/errors); Maven validate passed with zero Checkstyle violations. - BE: clang-format 16 passed for all 18 affected C++ files. All changed production translation units and the focused row-id fetcher, FileScannerV2, Parquet, and ORC test translation units compiled with ASAN enabled. - Validation limitation: BE test execution is pending CI. The full local test build is blocked in unchanged Lance reader code because available Lance libraries lack the branch-required FTS patch APIs. The external Iceberg regression suite was not run locally. - Behavior changed: Yes. Supported TopN second-phase fetches use FileScannerV2 when enabled. - Does this need documentation: No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
