Caideyipi opened a new pull request, #18918:
URL: https://github.com/apache/iotdb/pull/18918
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Description
Large tablet transfers can encounter avoidable receiver stalls in two
independent memory-control paths:
- When write memory is rejected, an insert holding the region write lock can
wait for a different TsFileProcessor in the same region to flush. The
asynchronous flush submitter needs that lock too. Mark the current memtable for
flushing so the writer can finish and release the lock. Admission still checks
the existing hard memtable limit, and normal flush selection still prefers the
largest memtables. This also handles a first tablet whose memory has been
reserved before the memtable contains data.
- RPC buffers skip shrinking for requests at or below their initial
capacity, including flush resets, retaining oversized arrays and their pool
reservations. Include these requests in the existing shrink policy while
preserving the initial capacity and shrink cooldown. If optional 1.5x growth
does not fit, reserve only the required frame size before waiting or failing;
successful fallback does not increment allocation-failure metrics.
## Validation
- Regression tests reproduced the admission and RPC allocation failures
before the fixes.
- English full-reactor build and 133 selected tests passed: 103 Service-RPC
tests, 4 SystemInfo tests, and 26 TsFileProcessor tests.
- Chinese full-reactor build and 44 targeted tests passed:
AutoResizingBufferTest, SystemInfoTest, and TsFileProcessorTest. Both locale
builds covered all 53 reactor modules, including main and test compilation,
with Checkstyle and Spotless checks.
- The broader Chinese Service-RPC run exposed two existing English-only
message assertions in `TElasticFramedTransportTest.testSingularSize` and
`DeepCopyRpcTransportFactoryTest.testIndependentMaxFrameSizeConfigurations`.
The buffer tests passed in that run.
- The original full historical-sync workload has not been rerun; throughput
improvement still needs workload verification.
This PR has:
- [x] been self-reviewed.
- [x] added comments explaining the intent of the changes.
- [x] added regression tests covering memory admission, hard-limit
rejection, frame integrity, capacity reuse between connections, and allocation
accounting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]