rionmonster commented on code in PR #2265:
URL: https://github.com/apache/fluss/pull/2265#discussion_r2673645505
##########
fluss-lake/fluss-lake-iceberg/src/test/java/org/apache/fluss/lake/iceberg/maintenance/IcebergRewriteITCase.java:
##########
@@ -186,6 +186,9 @@ void testLogTableCompaction() throws Exception {
t1, t1Bucket, ++i, true,
Collections.singletonList(row(1, "v1"))));
checkFileStatusInIcebergTable(t1, 3, false);
+ // Ensure tiering job has fully processed the previous writes
+ assertReplicaStatus(t1Bucket, i);
Review Comment:
@luoyuxia
I've been tinkering with a few different approaches to handling this and it
appears to entirely be related to the asynchronous processes in place (e.g.,
tiering, datalake freshness, etc.). I tried to avoid introducing any explicit
delays such as `Thread.sleep(...)` calls as those generally feel more like
addressing an underlying symptom as opposed to the cause.
I just recently ran a test through 500 and 1000 iterations respectively with
the `POLL_TIERING_TABLE_INTERVAL` and `TABLE_DATALAKE_FRESHNESS` set to 100ms
(from their original 500ms values). This appeared to address the flakiness.
Do this think this would be sufficient or are you aware of any additional
configurations that we might be able to use within the context of this test to
force flushes to help with consistency in some of these asynchronous processes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
