github-actions[bot] commented on code in PR #66825: URL: https://github.com/apache/doris/pull/66825#discussion_r3842933259
########## be/src/exec/pipeline/report_exec_status_size.h: ########## @@ -0,0 +1,47 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +#pragma once + +#include <gen_cpp/FrontendService_types.h> + +#include "common/status.h" +#include "util/thrift_util.h" + +namespace doris { + +inline bool report_transfers_external_file_ownership(const TReportExecStatusParams& params) { + return params.__isset.hive_partition_updates || params.__isset.iceberg_commit_datas || + params.__isset.mc_commit_datas || params.__isset.paimon_commit_messages; +} + +inline Status validate_report_exec_status_size(const TReportExecStatusParams& params, + size_t thrift_limit) { + ThriftSerializer serializer(false, 256); + uint32_t serialized_size = 0; + uint8_t* buffer = nullptr; + RETURN_IF_ERROR(serializer.serialize(¶ms, &serialized_size, &buffer)); Review Comment: [P1] Count the final report without materializing another full copy `params` already holds the assembled final report, but this new preflight serializes it into a growing `TMemoryBuffer` on the query-attached callback. That report path owns no admission token, and the default FE client uses streaming `TBufferedTransport`, so a valid report just below the 100 MiB receiver limit can newly need another nearly report-sized allocation, cross a tighter query/process hard limit, and cancel before the RPC. Please use an allocation-free counting transport/protocol (or explicitly admit this serialized image) and add a hard-limit regression where the retained report fits but the extra full image does not. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
