SEPURI-SAI-KRISHNA commented on issue #13330:
URL: https://github.com/apache/seatunnel/issues/13330#issuecomment-6999482088

   @SEZ9 thanks. The PR is #13332. Approach per case:
   
   1. Mixed types: bounds are grouped by type bracket, and each bracket gets 
its own ranges, so no range spans two types.
   2. Missing, null, NaN: only documents with a usable value are sampled. One 
more split reads the rest: `{$or: [{key: null}, {key: NaN}, {key: {$type: 
[types without a range]}}]}`. Each part can use an index on the key.
   3. `match.query`: combined with each bound by `$and` instead of merged.
   4. Dotted key: the value is read along the path.
   
   Tests: `SamplingSplitStrategyTest` has a case per row: the exact split 
filters for cases 1 to 3, with `match.query` in every split, and the value read 
along a dotted path for case 4. That every document is read exactly once is 
checked by `MongodbIT` (the exact rows written, on Zeta, Flink and Spark) and 
by the MongoDB 8.0.4 runs in the PR table.
   
   1 and 2 share the bracket and catch-all code, so I kept them together. 3 and 
4 are small and independent; I can split them into their own commits or PRs if 
that is easier to review.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to