JackieTien97 opened a new pull request, #19269:
URL: https://github.com/apache/iotdb/pull/19269

   ## Description
   
   Queries with very large time-partition requests can stay in front-end 
analysis while holding reservations from the operator memory pool. The Thrift 
buffer computes 1.5x growth using `int`; above roughly 1.33 GiB this overflows, 
so each small field write can copy the entire buffer while increasing its 
capacity by only a few bytes. The uncompressed frame limit was only checked at 
`flush()`, after serialization had already completed.
   
   This change:
   - Computes buffer growth and write positions without integer overflow and 
bounds them by the Java array limit.
   - Rejects oversized uncompressed frames before allocating or copying the 
next write, and closes the transport. Compressed transports retain their 
existing limit on the compressed frame size.
   - Checks the query deadline before enumerating time partitions and 
periodically during enumeration, allowing the existing Coordinator cleanup to 
release front-end reservations.
   
   ## Reproduction
   
   On `dev/1.3` at `013622816a`, a real 1,500,000,000-byte buffer grew to 
1,500,000,008, 1,500,000,016, and 1,500,000,024 bytes for three successive 
8-byte requests. Each request copied the entire array. With the fix, the first 
request grows the buffer once and the following requests reuse it.
   
   Regression tests also fail on the base revision for delayed frame-size 
rejection, write-position overflow, and missing partition-enumeration timeout 
checks. The query regression uses the real SQL parser, Coordinator, Analyzer, 
and generated ConfigNode Thrift serializer with test schema/partition fetchers 
and an in-memory transport. It checks that three successive oversized partition 
requests stop at the configured frame limit and fully restore operator-pool 
memory. This is a controlled reproduction of the code path, not a replay of the 
original production SQL or a live-cluster test.
   
   ## Validation
   
   ```sh
   mvn -B clean test -pl iotdb-core/datanode -am \
     
-Dtest=AutoResizingBufferTest,TElasticFramedTransportTest,RpcUtilsTest,UrlUtilsTest,TSStatusCodeTest,QueryTimePartitionTest,QueryPartitionMemoryTest,AnalyzeTest,QueryExecutionRetryTest
 \
     -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false
   ```
   
   53 distinct tests passed, including normal frame boundaries, 
compressed-frame compatibility, buffer reuse/shrinking, SQL analysis, query 
retry, and memory cleanup. Checkstyle and Spotless checks passed. The 
large-array reproduction was run separately with a bounded 5 GiB JVM heap so 
normal unit tests do not need multi-gigabyte allocations.
   
   - [x] Self-reviewed.
   - [x] Added regression tests.
   - [x] Added comments explaining the overflow, early rejection, and timeout 
behavior.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to