Gabriel39 opened a new pull request, #70205:
URL: https://github.com/apache/doris/pull/70205
### What problem does this PR solve?
Related PR: #70185
Boolean `_source` values such as `1.25`, integers outside the 32-bit range,
and
objects can reach incompatible RapidJSON getters. `IsNumber()` does not imply
`IsInt()`, and a non-numeric source value is not necessarily a string.
Validate
the exact types before decoding scalar and array Booleans, returning the
existing scan errors for unsupported values. Keep the existing numeric `0`
and
`1` representations.
Date mappings with an explicit format also require local predicate evaluation
in source mode. For example, `yyyy-MM-dd||epoch_millis` permits an epoch
that is
January 1 in ES's UTC index but January 2 in a `+08:00` session. Pushing the
UTC
predicate can return rows that violate the local predicate or discard
matching
rows. Classify these mappings as source-mode unsafe, retaining local
evaluation
through the existing planning path. Date docvalues remain eligible.
The regression fixture now checks both cross-day date predicates while
varying
scanner and parallel-scroll settings. No recorded golden results are changed.
### Release note
Fix incorrect date filtering and unsafe Boolean value access in Elasticsearch
source scans. Source-mode date filters may read more remote rows to preserve
session-local results.
### Check List (For Author)
- Test:
- ES connector unit tests: 164 tests, 0 failures/errors, 1 environment
skip.
The new explicit-format date test failed before the fix and passed after
it.
- FE Checkstyle and clang-format 16 checks passed.
- clang-tidy checked the modified BE source and test.
- A RapidJSON probe reproduced the original `GetInt()` assertion for
`1.25`.
- Full BE unit tests and the live ES regression suite were not completed
locally; those remain for CI.
- Behavior changed: Yes. Unsupported Boolean JSON values return scan errors;
source-mode date filters are evaluated locally without changing docvalue
eligibility.
- Does this need documentation: No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
