Gabriel39 opened a new pull request, #69333: URL: https://github.com/apache/doris/pull/69333
### What problem does this PR solve? Forward-port the complete changes from #69320 to `master`, including all review fixes through `a73b75002fa40f3df7c8189d8fe3560382d2e13a`. Add `LoadTableReader` so supported traditional file loads use FileScannerV2 with the existing FE/Thrift source and destination tuples. Source defaults, pre-filters, destination conversion, strict/nullability checks, and flexible partial-update bitmaps run in the table-reader finalization hook. CSV/JSON malformed records are rejected with rollback and load counters. JSON pipes consume every message until EOF; empty arrays and invalid configured roots contribute to the load quality gate without losing valid siblings. Master-specific adaptations: - Preserve master's ADBC/Fluss V2 query selection and Lance exclusion. - Match master's single-argument `TableReader::refresh_conjuncts()` and `RowDescriptor` constructors, and use `SlotDescriptor::type()`. - Omit the physical-split override and shared split-cache field, which are absent on master. Loads retain the existing FE split boundaries. Avro and Arrow IPC loads keep their existing path. No FE or Thrift changes. This port is independent of #69312. ### Validation - Passed the repository clang-format 16 check, covering all 17 changed C++ source/header files. - Passed `build-support/check-build-hygiene.sh` and `git diff --cached --check`. - Passed focused Clang syntax checks for all 11 changed C++ translation units and `table_reader.cpp`, using this master's generated headers and UT build configuration. The standalone scanner check suppresses two pre-existing unused-constant warnings from master. - Carried over the source PR's BE unit tests, JSON pipe tests, and Stream Load regression fixtures. - `run-be-ut.sh` completed configuration and started the full build. The fresh build requires about 14,750 targets, so it was stopped and replaced with the focused syntax checks above. No master runtime unit-test or cluster-regression pass is claimed; full build and runtime validation remain for CI. ### Release note Route supported traditional file loads through FileScannerV2 with a table reader for load conversion and validation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
