Tian Jiang created IOTDB-1856:
---------------------------------
Summary: Improve raft log size check
Key: IOTDB-1856
URL: https://issues.apache.org/jira/browse/IOTDB-1856
Project: Apache IoTDB
Issue Type: Improvement
Components: Core/Cluster
Reporter: Tian Jiang
Attachments: image-2021-10-18-09-51-15-583.png
To avoid log-append failures caused by oversized log entries, the size of each
entry is checked to ensure that it will fit the flush buffer before it is
appended to the local log. However, the current implementation has two
drawbacks:
# The size check falls in a synchronized block, which means concurrent requests
cannot be checked in parallel, and other requests will be significantly blocked
if the request is rather large.
# The size check directly serializes an entry into a ByteBuffer and uses it to
get the size of the entry, but the buffer is discarded immediately after the
check, so both CPU computation and memory allocation are wasted during the
serialization are wasted.
!image-2021-10-18-09-51-15-583.png|thumbnail!
To resolve the two aforementioned problems, we need:
# Move the size check out of the synchronized block.
# Use a serialization-free way to get the size of an entry, or save the
serialization result and reuse it during log persistency and log replication.
If the second one is adopted, the serialization result should be freed in time
to avoid a large memory footprint.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
