SEPURI-SAI-KRISHNA commented on issue #13330:
URL: https://github.com/apache/seatunnel/issues/13330#issuecomment-6999482088
@SEZ9 thanks. The PR is #13332. Approach per case:
1. Mixed types: bounds are grouped by type bracket, and each bracket gets
its own ranges, so no range spans two types.
2. Missing, null, NaN: only documents with a usable value are sampled. One
more split reads the rest: `{$or: [{key: null}, {key: NaN}, {key: {$type:
[types without a range]}}]}`. Each part can use an index on the key.
3. `match.query`: combined with each bound by `$and` instead of merged.
4. Dotted key: the value is read along the path.
Tests: `SamplingSplitStrategyTest` has a case per row: the exact split
filters for cases 1 to 3, with `match.query` in every split, and the value read
along a dotted path for case 4. That every document is read exactly once is
checked by `MongodbIT` (the exact rows written, on Zeta, Flink and Spark) and
by the MongoDB 8.0.4 runs in the PR table.
1 and 2 share the bracket and catch-all code, so I kept them together. 3 and
4 are small and independent; I can split them into their own commits or PRs if
that is easier to review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]