JackieTien97 commented on code in PR #18660:
URL: https://github.com/apache/iotdb/pull/18660#discussion_r4033118789
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/conf/DataNodeMemoryConfig.java:
##########
@@ -624,6 +627,13 @@ private void initQueryEngineMemoryAllocate(
setQueryThreadCount(
Integer.parseInt(
properties.getProperty("query_thread_count",
Integer.toString(getQueryThreadCount()))));
+
+ tableQueryDeviceEntryBatchSizeInBytes =
+ Long.parseLong(
+ properties.getProperty(
+ "table_query_device_entry_batch_size_in_bytes",
+ Long.toString(
+ operatorsMemoryManager.getTotalMemorySizeInBytes() /
queryThreadCount / 4)));
Review Comment:
[P2] Apply the RPC payload cap when initializing without a properties file
When neither a properties file nor an SPI provider is available,
`IoTDBDescriptor` only calls `memoryConfig.init(new TrimProperties())`. This
assignment therefore takes effect without
`loadTableQueryDeviceEntryBatchSize()` applying the `dn_thrift_max_frame_size -
1024` cap. The same threshold also controls `DeviceEntryDiskSpiller` segment
sizes.
In a fresh JVM with `-Xms64m -Xmx8g -XX:ActiveProcessorCount=1`, I
reproduced a batch threshold of `124429687` bytes while the configured default
frame limit was `67108864`. A component-level probe generated a `68157862`-byte
spill segment; an IoTDB framed transport enforcing the default 64 MiB limit
rejected its frame header with `Frame size (68157862) larger than protect max
size (67108864)!`. A receiving DataNode using that limit cannot fetch the
segment, so a spilled distributed query can fail.
Please apply the same cap normalization on the no-properties path and add a
fresh-JVM regression test with no configuration source and a computed batch
default above the frame limit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]