github-actions[bot] commented on code in PR #61828: URL: https://github.com/apache/doris/pull/61828#discussion_r3449678870
########## regression-test/suites/query_profile/test_profile_collection_completed.groovy: ########## @@ -0,0 +1,89 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +/** + * Verifies that the "Is Profile Collection Completed: true" field + * appears in the profile text once all BE fragment profiles have + * been collected (or the collection timeout has elapsed). + */ +suite("test_profile_collection_completed") { + + def getProfileText = { String queryId -> + def addr = context.config.feHttpAddress + def user = context.config.feHttpUser ?: "root" + def pass = context.config.feHttpPassword ?: "" + def encoding = Base64.getEncoder().encodeToString("${user}:${pass}".getBytes("UTF-8")) + def conn = new URL("http://${addr}/api/profile/text?query_id=${queryId}").openConnection() + conn.setRequestMethod("GET") + conn.setRequestProperty("Authorization", "Basic ${encoding}") + conn.setConnectTimeout(5000) + conn.setReadTimeout(10000) + try { + return conn.getInputStream().getText() + } catch (Exception e) { + return "" + } + } + + // Poll until the completion marker appears, or timeout after 10 s. + def waitForCompletionMarker = { String queryId -> + def marker = "Is Profile Collection Completed: true" + def maxAttempts = 33 // 33 × 300 ms ≈ 10 s Review Comment: This wait budget is just under the production timeout path this test says it covers. The loop sleeps `33 * 300ms ~= 9.9s`, but `shouldStoreToStorage()` only enters the timeout branch after `profile_waiting_time_for_spill_seconds` is exceeded (`>` 10s by default), and `ProfileManager` only evaluates it on its daemon interval. If BE profile collection is slow enough to rely on the timeout path, the marker is not guaranteed to appear before this assertion. Please either wait longer than the spill timeout plus the manager interval, or lower/restore the FE config in this suite so the timeout path is deterministic. ########## fe/fe-core/src/main/java/org/apache/doris/common/profile/Profile.java: ########## @@ -567,6 +567,7 @@ public boolean shouldStoreToStorage() { } if (!hasReportingProfile) { + summaryProfile.markCollectionCompleted(); Review Comment: This still can publish the marker before all BEs have reported. `ExecutionProfile.isCompleted()` compares `fragmentProfile.getChildList().size()` with `fragmentIdBeNum`, but one BE report adds multiple children to that fragment profile: BE `collect_realtime_profile()` sends the fragment-level profile first and then each pipeline profile, and FE `updateProfile()` adds a child for every `TDetailedReportParams`. With two assigned backends and one backend reporting fragment-level + one pipeline profile, the child count reaches `fragmentIdBeNum` and all existing children can be done, so this branch marks collection complete and the same storage pass can release the profile before the second backend report is merged. Please gate this on distinct reporting backends, for example the `multiBeProfile` backend-address set, rather than total child count. ########## regression-test/suites/query_profile/test_profile_collection_completed.groovy: ########## @@ -0,0 +1,89 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +/** + * Verifies that the "Is Profile Collection Completed: true" field + * appears in the profile text once all BE fragment profiles have + * been collected (or the collection timeout has elapsed). + */ +suite("test_profile_collection_completed") { + + def getProfileText = { String queryId -> + def addr = context.config.feHttpAddress + def user = context.config.feHttpUser ?: "root" Review Comment: This direct profile fetch is not cloud-aware. Existing profile-test helpers switch to `feCloudHttpUser`/`feCloudHttpPassword` when `context.config.isCloudMode()` is true, but this suite always authenticates with `feHttpUser`/`feHttpPassword`. In cloud regression configs with different profile credentials, the polling loop will swallow the 401/403 as `""` until the marker assertion fails. Please use the existing `ProfileAction`/HTTP helper or mirror its cloud-mode credential selection. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
