Gabriel39 opened a new pull request, #70604:
URL: https://github.com/apache/doris/pull/70604

   ### Background
   
   The native MongoDB Catalog on master already provides single-cursor reads 
and first-release BSON SerDes. SQL filters and source projection have remained 
local, and the remaining M1 type and pushdown contracts need end-to-end 
qualification.
   
   ### What problem does this PR solve?
   
   Issue Number: #70251 (D04b and D05)
   Related PR: #70327 (native read path), #70456 (M0 acceptance)
   PR Type: Feature
   
   Problem Summary:
   
   Complete the D04b qualification coverage and connect conservative 
predicate/LIKE candidates and column pruning to the native MongoDB reader. 
Every SQL predicate remains a Doris residual; the default strict type policy, 
strict SQL casts and standard views keep filtering local. This PR does not 
complete the separate D06 or later snapshot/parallel-scan work.
   
   ### Details
   
   - **Change map:** 449 added production lines, 1,187 
test/fixture/generated-result lines, and 47 documentation lines across 19 
files. The production changes comprise a connector-local translator and scan 
planning, three session controls, two optional descriptor fields, and native 
find options/validation. Most additions qualify types and compare execution 
modes.
   - **Scope:** D04b covers Decimal128 extremes, exact integer/floating-point 
behavior, UTC dates and scales, binary ownership/subtypes, missing/null, 
recursive arrays/structs, rollback and existing document limits. D05 covers 
typed scalar-root comparisons, positive IN/BETWEEN, IS NOT NULL, safe AND/OR, 
necessary-literal LIKE candidates, root projection and time-series Date ranges.
   - **Design:** Candidates are classified conservatively as SUPERSET; 
unsupported expressions remain local, and the EXACT category is reserved for 
proven equivalence. No conjunct is consumed through either SPI path. CASTs are 
rejected before the neutral converter can erase them. The translator reconciles 
the schema/expression SPI date-scale and decimal-width representations, 
including FE's DECIMAL128 name. Native BSON is bounded and validated before 
client acquisition; views cannot receive a predicate/collation override. 
Complete root projection retains residual columns and scalar-ancestor errors.
   - **Usage:** Use `SET` or `SET_VAR` with 
`enable_mongodb_predicate_pushdown`, `enable_mongodb_like_pushdown` and 
`enable_mongodb_projection_pushdown`. Predicate candidates require the explicit 
catalog policy `mongodb.type_mismatch=null`.
   - **Default behavior:** All three switches default to true. The default 
`type_mismatch=error` policy preserves local filtering; 
`enable_strict_cast=true` and views also preserve local filtering. Source 
projection is eligible independently. DOUBLE/NaN comparisons, NOT/!=/NOT IN, 
mismatch-sensitive IS NULL, functions, casts, nested predicate paths and 
existing direct VARBINARY operator restrictions stay local. LIKE keeps the 
complete SQL residual. LIMIT, aggregation, sort, joins, snapshots and parallel 
scans are unchanged.
   - **Compatibility:** The connector SPI and stored schemas are unchanged. 
Descriptor fields 23/24 are optional; absent fields preserve the prior full 
scan. The touched descriptor is absent from the four supported release 
branches, so no field id is reused. Qualification uses matching FE/BE versions.
   
   ### Testing
   
   All testing below ran locally against MongoDB 8.0.20 with authentication 
enabled and an ASAN BE:
   
   - 59 MongoDB connector FE tests, 25 connector-session/forwarding tests and 
55 BSON/native-reader BE tests passed.
   - All three MongoDB SQL suites passed: 3 suites, 0 failed, 0 fatal, 0 
skipped. This includes the existing native and M0 acceptance suites plus the 
new M1 suite.
   - The generated M1 output contains 38 identical sorted result pairs with 
predicate/LIKE/projection disabled and enabled. It covers precision, 
NULL/arrays, casts, collations, residual LIMIT, aggregates/joins, views and 
time-series reads. The completed Profile check reports `MongoRowsRead` of 10 
with candidates disabled and 1 with them enabled.
   - Source profiling independently confirmed actual BSON int64, Decimal128 and 
Date filters, projection, simple collection collation, and view reads without a 
collation override.
   - FE and ASAN BE builds passed; the final MongoDB plugin was subsequently 
packaged/tested and deployed from its final source. Full FE Checkstyle and the 
final affected-core-file Checkstyle run passed with zero violations. All 
affected C/C++ files pass clang-format 16, build hygiene and clang-tidy.
   - Red/green checks reproduced missing candidate planning, acceptance of 
malformed BSON descriptors, missing strict-cast gating, command-budget gaps and 
schema/expression type representation gaps before the corresponding 
implementations.
   
   ### Release note
   
   Native MongoDB scans can prune source fields and apply conservative 
WHERE/LIKE candidates under the explicit null-on-type-mismatch policy. SQL 
residuals remain active, and default strict scans and views preserve local 
filtering. New session switches allow statement-level comparisons and disabling 
these optimizations.
   
   ### Check List (For Author)
   
   - Behavior changed: Yes (see Details).
   - Does this need documentation: Yes; the connector and fixture READMEs are 
updated here.
   - Pre-submission status: Ready. Local UT, authenticated SQL regression, 
builds and style/static checks passed; CI is pending.
   
   ### AI Disclosure
   
   - Tool: Codex (GPT-6).
   - Generated: Implementation, tests, fixture, documentation and this 
description.
   - Human review: Pending maintainer review.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to