LiebingYu commented on code in PR #1749:
URL: https://github.com/apache/fluss/pull/1749#discussion_r2380710973
##########
fluss-server/src/main/java/org/apache/fluss/server/log/LogLoader.java:
##########
@@ -129,16 +165,106 @@ public LoadedLogOffsets load() throws IOException {
* overflow
*/
private Tuple2<Long, Long> recoverLog() throws IOException {
- // TODO truncate log to recover maybe unflush segments.
+ if (!isCleanShutdown) {
+ List<LogSegment> unflushed =
+ logSegments.values(recoveryPointCheckpoint,
Long.MAX_VALUE);
+ int numUnflushed = unflushed.size();
+ Iterator<LogSegment> unflushedIter = unflushed.iterator();
+ boolean truncated = false;
+ int numFlushed = 1;
+
+ while (unflushedIter.hasNext() && !truncated) {
+ LogSegment segment = unflushedIter.next();
+ LOG.info(
+ "Recovering unflushed segment {}. {}/{} recovered for
bucket {}",
+ segment.getBaseOffset(),
+ numFlushed,
+ numUnflushed,
+ logSegments.getTableBucket());
+
+ int truncatedBytes = -1;
Review Comment:
This logic also follow Kafka. From my perspective, data loss is unlikely
because the data is stored in multiple replicas. Once the file is truncated to
the correct position, it can synchronize the latest data from the leader. If
truncation is not carried out, the file appears to be unrecoverable, and if the
host machine becomes the leader afterward, unforeseen problems might occur.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
