chrisdutz commented on code in PR #19108:
URL: https://github.com/apache/iotdb/pull/19108#discussion_r4523587640
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/flush/MemTableFlushTask.java:
##########
@@ -141,15 +141,30 @@ public MemTableFlushTask(
writer.getFile().getName());
}
+ /**
+ * Temporary memory of a flush: the bounded IO queue holds up to {@code
ioTaskQueueSize} encoded
+ * series of average size. With no more series than queue slots the queue
can't hold more than the
+ * memtable, so the estimate is capped at {@code memSize}. Without the cap a
memtable of 3 series
+ * reserved 3.3 times its size, which can exceed the whole MemTable budget
and make the reject
+ * threshold negative.
+ */
+ static long estimateTemporaryMemory(
+ long memSize, int seriesNumber, int ioTaskQueueSize, boolean
encodingFasterThanIo) {
+ if (!encodingFasterThanIo || seriesNumber == 0) {
+ return 0;
+ }
+ return seriesNumber <= ioTaskQueueSize ? memSize : memSize / seriesNumber
* ioTaskQueueSize;
Review Comment:
Thanks, good catch. I could reproduce it: with I/O stalled until the queue
is full, one device with nine `DOUBLE` series (`PLAIN`, `UNCOMPRESSED`, 60,000
points each) gives 9,441,072 bytes of queued chunk buffers
(`ChunkWriterImpl.pageBuffer`) against a `memSize()` of 8,640,000. I only got
numbers close to yours with the time column `PLAIN` as well; with the default
`TS_2DIFF` time encoder the buffers stay at about half of the reservation.
The data written into the buffers matches `memSize()` almost exactly
(8,649,045 bytes). The difference is the spare capacity the buffers keep after
growing by doubling. The old estimate doesn't cover it either. It only came out
ahead in this case because ten queue slots for nine series add about 11%. With
more series than slots, both versions reserve the same amount and both come up
short:
| Series × points | Reservation: master / this PR so far | Queued buffers |
Buffers / reservation |
|---|---|---|---|
| 9 × 60,000 | 9,600,000 / 8,640,000 | 9,441,072 | 0.98 / 1.09 |
| 30 × 60,000 | 9,600,000 (both) | 10,490,080 | 1.09 |
| 300 × 6,000 | 960,000 (both) | 1,311,540 | 1.37 |
Since the buffers' capacity can approach twice their content, the estimate
now doubles the queued share. The cap on the raw share stays, so the
reservation is at most twice the memtable, instead of up to ten times (the
queue size divided by the number of series) before this PR.
`MemTableFlushTaskTest.testReservationCoversQueuedChunkBuffersWithStalledIo`
is the regression test you asked for. It stalls the I/O consumer, waits until
the encoder has filled the queue, and checks that the reservation actually
taken from `SystemInfo` covers the capacity of the queued chunk buffers, for 9
× 60,000, 1 × 60,000 and 300 × 6,000 (time and values `PLAIN`). It fails
without the factor (9,441,072 B of buffers against 8,640,000 B reserved) and
passes with it.
One trade-off: the flushing memtable keeps counting towards the reject
threshold until the flush ends, and the reservation lowers that threshold. In
small memory configurations with few series per memtable, a 2× reservation can
therefore put writes into reject status for the duration of a flush. If you
think that matters, I could make the factor configurable (e.g.
`temporary_memory_factor_for_flushing`, default 2.0), so such setups can lower
it. I left it out to keep this PR small.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]