liyuheng55555 opened a new pull request, #12955: URL: https://github.com/apache/iotdb/pull/12955
In the current Write-Ahead Logging (WAL) mechanism, write operations with the same sequence number may be split into multiple WAL entries and written into different WAL files. IoTConsensus constructs synchronization batches from WAL files for synchronization between replicas. Taking a write operation with sequence number `a` as an example, the sync batch constructed by IoTConsensus for operation `a` needs to **completely** include all WAL entries with sequence number `a`. The problem lies in how to determine **completely**. As the flushing of WAL is asynchronous, IoTConsensus currently waits until a write operation with sequence number `a+1` arrives, or some other operation with a sequence number of `-1` (indicating no need for synchronization between replicas) arrives, before IoTConsensus can confirm that operation `a` can be fully read from the WAL file. **This causes the last write operation to never be synchronized** (Admittedly, if operation `a` can enter pendingEntries, it can be synchronized normally, but pendingEntries is essentially an engineering optimization and should be ignored when thinking outside of IoTConsensus module). In order to allow the last write operation to be synchronized normally, this PR adds an additional special WAL with a sequence number of -1 after each batch of WAL entries with the same sequence number is written, so that IoTConsensus can immediately know this after the current write operation is completed. See **BatchDoneNode** for details. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
