Gabriel39 opened a new pull request, #70604: URL: https://github.com/apache/doris/pull/70604
### Background The native MongoDB Catalog on master already provides single-cursor reads and first-release BSON SerDes. SQL filters and source projection have remained local, and the remaining M1 type and pushdown contracts need end-to-end qualification. ### What problem does this PR solve? Issue Number: #70251 (D04b and D05) Related PR: #70327 (native read path), #70456 (M0 acceptance) PR Type: Feature Problem Summary: Complete the D04b qualification coverage and connect conservative predicate/LIKE candidates and column pruning to the native MongoDB reader. Every SQL predicate remains a Doris residual; the default strict type policy, strict SQL casts and standard views keep filtering local. This PR does not complete the separate D06 or later snapshot/parallel-scan work. ### Details - **Change map:** 449 added production lines, 1,187 test/fixture/generated-result lines, and 47 documentation lines across 19 files. The production changes comprise a connector-local translator and scan planning, three session controls, two optional descriptor fields, and native find options/validation. Most additions qualify types and compare execution modes. - **Scope:** D04b covers Decimal128 extremes, exact integer/floating-point behavior, UTC dates and scales, binary ownership/subtypes, missing/null, recursive arrays/structs, rollback and existing document limits. D05 covers typed scalar-root comparisons, positive IN/BETWEEN, IS NOT NULL, safe AND/OR, necessary-literal LIKE candidates, root projection and time-series Date ranges. - **Design:** Candidates are classified conservatively as SUPERSET; unsupported expressions remain local, and the EXACT category is reserved for proven equivalence. No conjunct is consumed through either SPI path. CASTs are rejected before the neutral converter can erase them. The translator reconciles the schema/expression SPI date-scale and decimal-width representations, including FE's DECIMAL128 name. Native BSON is bounded and validated before client acquisition; views cannot receive a predicate/collation override. Complete root projection retains residual columns and scalar-ancestor errors. - **Usage:** Use `SET` or `SET_VAR` with `enable_mongodb_predicate_pushdown`, `enable_mongodb_like_pushdown` and `enable_mongodb_projection_pushdown`. Predicate candidates require the explicit catalog policy `mongodb.type_mismatch=null`. - **Default behavior:** All three switches default to true. The default `type_mismatch=error` policy preserves local filtering; `enable_strict_cast=true` and views also preserve local filtering. Source projection is eligible independently. DOUBLE/NaN comparisons, NOT/!=/NOT IN, mismatch-sensitive IS NULL, functions, casts, nested predicate paths and existing direct VARBINARY operator restrictions stay local. LIKE keeps the complete SQL residual. LIMIT, aggregation, sort, joins, snapshots and parallel scans are unchanged. - **Compatibility:** The connector SPI and stored schemas are unchanged. Descriptor fields 23/24 are optional; absent fields preserve the prior full scan. The touched descriptor is absent from the four supported release branches, so no field id is reused. Qualification uses matching FE/BE versions. ### Testing All testing below ran locally against MongoDB 8.0.20 with authentication enabled and an ASAN BE: - 59 MongoDB connector FE tests, 25 connector-session/forwarding tests and 55 BSON/native-reader BE tests passed. - All three MongoDB SQL suites passed: 3 suites, 0 failed, 0 fatal, 0 skipped. This includes the existing native and M0 acceptance suites plus the new M1 suite. - The generated M1 output contains 38 identical sorted result pairs with predicate/LIKE/projection disabled and enabled. It covers precision, NULL/arrays, casts, collations, residual LIMIT, aggregates/joins, views and time-series reads. The completed Profile check reports `MongoRowsRead` of 10 with candidates disabled and 1 with them enabled. - Source profiling independently confirmed actual BSON int64, Decimal128 and Date filters, projection, simple collection collation, and view reads without a collation override. - FE and ASAN BE builds passed; the final MongoDB plugin was subsequently packaged/tested and deployed from its final source. Full FE Checkstyle and the final affected-core-file Checkstyle run passed with zero violations. All affected C/C++ files pass clang-format 16, build hygiene and clang-tidy. - Red/green checks reproduced missing candidate planning, acceptance of malformed BSON descriptors, missing strict-cast gating, command-budget gaps and schema/expression type representation gaps before the corresponding implementations. ### Release note Native MongoDB scans can prune source fields and apply conservative WHERE/LIKE candidates under the explicit null-on-type-mismatch policy. SQL residuals remain active, and default strict scans and views preserve local filtering. New session switches allow statement-level comparisons and disabling these optimizations. ### Check List (For Author) - Behavior changed: Yes (see Details). - Does this need documentation: Yes; the connector and fixture READMEs are updated here. - Pre-submission status: Ready. Local UT, authenticated SQL regression, builds and style/static checks passed; CI is pending. ### AI Disclosure - Tool: Codex (GPT-6). - Generated: Implementation, tests, fixture, documentation and this description. - Human review: Pending maintainer review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
