JackieTien97 commented on code in PR #18660:
URL: https://github.com/apache/iotdb/pull/18660#discussion_r4033118789


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/conf/DataNodeMemoryConfig.java:
##########
@@ -624,6 +627,13 @@ private void initQueryEngineMemoryAllocate(
     setQueryThreadCount(
         Integer.parseInt(
             properties.getProperty("query_thread_count", 
Integer.toString(getQueryThreadCount()))));
+
+    tableQueryDeviceEntryBatchSizeInBytes =
+        Long.parseLong(
+            properties.getProperty(
+                "table_query_device_entry_batch_size_in_bytes",
+                Long.toString(
+                    operatorsMemoryManager.getTotalMemorySizeInBytes() / 
queryThreadCount / 4)));

Review Comment:
   [P2] Apply the RPC payload cap when initializing without a properties file
   
   When neither a properties file nor an SPI provider is available, 
`IoTDBDescriptor` only calls `memoryConfig.init(new TrimProperties())`. This 
assignment therefore takes effect without 
`loadTableQueryDeviceEntryBatchSize()` applying the `dn_thrift_max_frame_size - 
1024` cap. The same threshold also controls `DeviceEntryDiskSpiller` segment 
sizes.
   
   In a fresh JVM with `-Xms64m -Xmx8g -XX:ActiveProcessorCount=1`, I 
reproduced a batch threshold of `124429687` bytes while the configured default 
frame limit was `67108864`. A component-level probe generated a `68157862`-byte 
spill segment; an IoTDB framed transport enforcing the default 64 MiB limit 
rejected its frame header with `Frame size (68157862) larger than protect max 
size (67108864)!`. A receiving DataNode using that limit cannot fetch the 
segment, so a spilled distributed query can fail.
   
   Please apply the same cap normalization on the no-properties path and add a 
fresh-JVM regression test with no configuration source and a computed batch 
default above the frame limit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to