Gabriel39 opened a new pull request, #69723:
URL: https://github.com/apache/doris/pull/69723
### What problem does this PR solve?
File scans still maintain the legacy scanner, its readers and a second
capability/routing matrix. Route all file scans, file loads and external row-ID
fetches through `FileScannerV2`, and remove the unused V1 implementation and
tests.
- Keep `enable_file_scanner_v2` as a deprecated compatibility variable. Its
stored value no longer affects planning or scanning; FE sends `true` in the
existing Thrift field for older backends.
- Remove scanner-level capability checks. Factories and concrete readers
return errors for unsupported operations, including native Hudi delta-log
reads, query-only connector loads and Remote Doris Variant reads.
- Move standalone Iceberg position-delete reads to V2 `TableReader`, and
historical Push ingestion to `LoadTableReader`. Preserve Push conversion
expressions across ranges and compatibility with requests that omit the
file-column count.
- Retain shared writer, metadata, JDBC/JNI and Elasticsearch HTTP components
that surviving readers still use. Remove V1-only schema, partition, count and
decoder infrastructure.
- Update connector planning, nullable COUNT pushdown, documentation and
compatibility regression expectations.
Related PR: #69668
### Release note
File scans always use FileScannerV2. `enable_file_scanner_v2` is deprecated
and has no effect. Unsupported reader operations return an error instead of
selecting the legacy scanner.
### Check List (For Author)
- Test
- [x] Unit Test
- [x] Regression test
- [x] Manual test
Validation:
- Production BE and FE build, including connector plugin packages.
- 523 focused FE tests and FE Checkstyle.
- 593 distinct BE tests; the Remote Doris Flight suite was rerun
successfully with localhost excluded from the development proxy.
- clang-format 16, clang-tidy on changed lines, build hygiene and its full
self-test suite.
- Final Push fixture rerun passed after the test-data helper cleanup.
- Six standalone regression suites passed: Arrow Stream Load, HTTP
CSV/JSON/compression/header formats and schema inference.
- Manual SQL verified that setting the old option to false remains accepted
and SHOW VARIABLES marks it deprecated.
External Paimon regression expectations are updated for the no-op option and
will be exercised by the external CI environment.
The exhaustive CSV/JSON split tests retain their existing boundary
assertions after dropping the V1 parameterization. Four narrow
cognitive-complexity suppressions preserve these existing GTest-expanded
cross-product tests; new tests are decomposed instead.
- Behavior changed:
- [x] Yes. The scanner option is now a deprecated no-op; unsupported
operations fail in their reader.
- Does this need documentation?
- [x] Yes. The scanner design and review guide are updated in this PR.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
