github-actions[bot] commented on code in PR #62589: URL: https://github.com/apache/doris/pull/62589#discussion_r3241048647
########## regression-test/suites/query_p0/runtime_filter/rf_partition_pruning_type_matrix.groovy: ########## @@ -0,0 +1,219 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +import org.apache.doris.regression.action.ProfileAction + +suite("rf_partition_pruning_type_matrix", "nonConcurrent") { + sql "set enable_runtime_filter_prune=false" + sql "set enable_runtime_filter_partition_prune=true" + sql "set runtime_filter_wait_infinitely=true" + sql "set disable_join_reorder=true" + sql "set enable_profile=true" + sql "set profile_level=2" + sql "set parallel_pipeline_task_num=1" + + def profileAction = new ProfileAction(context) + def rfPruningCounterNames = ["TotalPartitionsForRFPruning", "PartitionsPrunedByRuntimeFilter"] + + def getProfileByToken = { String token -> + String profileContent = "" + for (int attempt = 0; attempt < 60; attempt++) { + List profileData = profileAction.getProfileList() + for (final def profileItem in profileData) { + if (profileItem["Sql Statement"].toString().contains(token)) { + profileContent = profileAction.getProfile(profileItem["Profile ID"].toString()) + break + } + } + if (profileContent != "" && rfPruningCounterNames.every { profileContent.contains(it) }) { + break + } + Thread.sleep(500) + } + return profileContent + } + + def extractCounterSum = { String profileText, String counterName -> + def values = (profileText =~ /-\s*${counterName}:\s*(\d+)/).collect { it[1].toLong() } + return values.isEmpty() ? 0L : values.sum() + } + + def assertPruningProfile = { String queryBody, String rfType, long expectedTotal, long expectedPruned -> + def token = UUID.randomUUID().toString() + def querySql = """ + SELECT /*+ SET_VAR(runtime_filter_type='${rfType}') */ "${token}", * + ${queryBody} + ORDER BY 2 + """ + sql querySql + def profile = getProfileByToken(token) Review Comment: This helper executes the join and then discards the result, so the type-matrix cases only prove that the profile counters are at least the expected values. A false-positive pruning bug can still pass here: for example, if an INT RANGE case prunes all four partitions instead of keeping the partition containing `dim_key = 150`, `PartitionsPrunedByRuntimeFilter >= 3` still succeeds while the query returns no matching fact row. Please have the helper assert the expected joined rows (or at least the expected fact ids) in addition to the pruning counters, so these cases catch over-pruning/data-loss regressions. This is distinct from the existing profile-polling/golden-file threads because the profile can be present and the counters can satisfy the current assertions while the query result is wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
