github-actions[bot] commented on code in PR #65903:
URL: https://github.com/apache/doris/pull/65903#discussion_r3629946315
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/Cast.java:
##########
@@ -273,10 +281,38 @@ public Expression withConstantArgs(Expression literal) {
@Override
public boolean isMonotonic(Literal lower, Literal upper) {
- // Both upward and downward casting of date types satisfy monotonicity.
- if (child().getDataType() instanceof DateLikeType && targetType
instanceof DateLikeType) {
+ DataType childType = child().getDataType();
+ if (!(childType instanceof DateLikeType && targetType instanceof
DateLikeType)) {
+ return false;
+ }
+
+ if (childType instanceof TimeStampTzType && targetType instanceof
DateTimeV2Type) {
+ return isTimeStampTzToDateTimeV2Monotonic(lower, upper);
+ }
+ return true;
+ }
+
+ private boolean isTimeStampTzToDateTimeV2Monotonic(Literal lower, Literal
upper) {
+ ZoneId timeZone;
+ try {
+ timeZone = DateUtils.getTimeZone();
+ } catch (DateTimeException e) {
+ return false;
+ }
+ if (timeZone.getRules().isFixedOffset()) {
return true;
}
- return false;
+ if (!(lower instanceof TimestampTzLiteral) || !(upper instanceof
TimestampTzLiteral)) {
+ return false;
+ }
+
+ // TimestampTzLiteral stores UTC civil fields. The cast renders those
instants in the
+ // session timezone, which moves backward at a fall-back transition.
+ Instant lowerInstant = ((TimestampTzLiteral)
lower).toJavaDateType().toInstant(ZoneOffset.UTC);
+ Instant upperInstant = ((TimestampTzLiteral)
upper).toJavaDateType().toInstant(ZoneOffset.UTC);
+ if (upperInstant.isBefore(lowerInstant)) {
+ return false;
+ }
+ return !DateUtils.hasFallbackTransitionInInstantRange(timeZone,
lowerInstant, upperInstant);
Review Comment:
**[P1] Check fall-backs after applying target-scale rounding**
This searches transitions using the original TIMESTAMPTZ bounds, but both FE
folding and BE `TimestampTzValue::to_datetime` round the UTC value to the
DATETIMEV2 target scale before applying the session zone. For `TIMESTAMPTZ(6)
-> DATETIMEV2(0)` in `America/New_York`, a partition `[05:00:00Z,
05:59:59.900000Z)` contains no raw transition, so this returns true;
nevertheless values from `05:59:59.500000Z` round to the `06:00Z` fall-back and
render as `01:00:00`, while earlier values reach `01:59:59`. Both endpoints
project to `01:00:00`, so static/RF pruning can build a singleton envelope and
drop a matching `01:30:00` row. Please apply the cast's target-scale rounding
before the transition test (or conservatively reject scale reduction) and cover
this case.
##########
regression-test/suites/datatype_p0/timestamptz/test_timestamptz_rf_partition_prune_dst.groovy:
##########
@@ -0,0 +1,89 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_timestamptz_rf_partition_prune_dst") {
+ sql "SET time_zone='America/New_York'"
+ sql "SET enable_nereids_planner=true"
+ sql "SET enable_fallback_to_original_planner=false"
+ sql "SET enable_runtime_filter_prune=false"
+ sql "SET runtime_filter_wait_infinitely=true"
+ sql "SET runtime_filter_mode='GLOBAL'"
+ sql "SET runtime_filter_type='IN_OR_BLOOM_FILTER'"
Review Comment:
**[P2] Pin an IN filter so this exercises RANGE RF pruning**
P0 regression enables fuzzy session variables, which can set
`runtime_filter_max_in_num=0`. With this one-row build side,
`IN_OR_BLOOM_FILTER` then becomes Bloom, and the BE RF partition pruner
deliberately skips Bloom filters for RANGE boundaries; `rf_prune_on` can
consequently return the expected rows without making the changed monotonicity
classification affect RANGE partition pruning. Please pin
`runtime_filter_max_in_num` to a positive value such as `1024` (as the existing
RF partition-pruning suite does), ideally with a pruning profile assertion.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/Cast.java:
##########
@@ -273,10 +281,38 @@ public Expression withConstantArgs(Expression literal) {
@Override
public boolean isMonotonic(Literal lower, Literal upper) {
- // Both upward and downward casting of date types satisfy monotonicity.
- if (child().getDataType() instanceof DateLikeType && targetType
instanceof DateLikeType) {
+ DataType childType = child().getDataType();
+ if (!(childType instanceof DateLikeType && targetType instanceof
DateLikeType)) {
+ return false;
+ }
+
+ if (childType instanceof TimeStampTzType && targetType instanceof
DateTimeV2Type) {
+ return isTimeStampTzToDateTimeV2Monotonic(lower, upper);
+ }
+ return true;
+ }
+
+ private boolean isTimeStampTzToDateTimeV2Monotonic(Literal lower, Literal
upper) {
+ ZoneId timeZone;
+ try {
+ timeZone = DateUtils.getTimeZone();
Review Comment:
**[P1] Avoid certifying with a different timezone-rule source**
This decision uses the FE JVM's `ZoneRules`, while the cast and RF endpoint
projection execute with cctz rules loaded independently on each BE from `TZDIR`
or `/usr/share/zoneinfo`; only the zone name is forwarded, and no tzdb-version
invariant is enforced. After a timezone-law update, FE can therefore see no
fall-back in a named-zone range and publish `MONOTONIC_INCREASING` while a BE
with different tzdata observes a backward jump, letting endpoint pruning drop
matching rows. Please keep non-fixed zones conservative unless rule parity is
enforced, or base the certificate on the same transition data used by execution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
