swuferhong commented on code in PR #3623:
URL: https://github.com/apache/fluss/pull/3623#discussion_r3586029729
##########
fluss-flink/fluss-flink-common/src/test/java/org/apache/fluss/flink/source/FlinkTableSourceBatchITCase.java:
##########
@@ -59,11 +59,14 @@
abstract class FlinkTableSourceBatchITCase extends FlinkTestBase {
static final String CATALOG_NAME = "testcatalog";
- static final String DEFAULT_DB = "defaultdb";
protected StreamTableEnvironment tEnv;
+ private String databaseName;
+ private boolean databaseCreated;
Review Comment:
Why this class need modify?
##########
fluss-server/src/main/java/org/apache/fluss/server/log/LogTablet.java:
##########
@@ -1196,17 +1231,29 @@ private List<LogSegment> deletableSegments(long
endOffset) {
// readers is in progress.
List<LogSegment> deletableSegments = new ArrayList<>();
List<LogSegment> logSegments = localLog.getSegments().values();
- // ignore the segments configured to be retained
- for (int i = 0; i < logSegments.size() - tieredLogLocalSegments; i++) {
- if (logSegments.get(i + 1).getBaseOffset() <= endOffset) {
- deletableSegments.add(logSegments.get(i));
- } else {
+ int tierProtectedStartIndex = logSegments.size() -
tieredLogLocalSegments;
+ long now = clock.milliseconds();
+
+ for (int i = 0; i < logSegments.size() - 1; i++) {
+ if (logSegments.get(i + 1).getBaseOffset() > endOffset) {
break;
}
+ if (i < tierProtectedStartIndex
+ || isSegmentExpired(now, logSegments.get(i), logTtlMs)) {
+ deletableSegments.add(logSegments.get(i));
Review Comment:
In the retention window, break as soon as the first non-deletable segment is
encountered, so that only a single contiguous prefix is deleted. If timestamps
are monotonic, this changes nothing.
##########
fluss-server/src/main/java/org/apache/fluss/server/log/LogTablet.java:
##########
@@ -556,9 +594,6 @@ public void updateRemoteLogSize(long remoteLogSize) {
public void updateRemoteLogEndOffset(long remoteLogEndOffset) {
if (remoteLogEndOffset > this.remoteLogEndOffset) {
this.remoteLogEndOffset = remoteLogEndOffset;
-
- // try to delete these segments already exist in remote storage.
- deleteSegmentsAlreadyExistsInRemote();
Review Comment:
Why can this inline cleanup be removed? If we remove it, local segments
won't be reclaimed even after they've already been moved to remote — they can
only be cleaned up by TTL. That makes our "retain 2 local segments"
(tieredLogLocalSegments) mechanism useless, and local disk will grow unbounded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
