bbejeck commented on code in PR #23086:
URL: https://github.com/apache/kafka/pull/23086#discussion_r3754399696
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java:
##########
@@ -199,13 +200,119 @@ long calculateRestoreTime(final long restoreEndTimeNs) {
private static final long DEFAULT_OFFSET_UPDATE_MS =
Duration.ofMinutes(5L).toMillis();
+ // Windows the probe reads back from each partition's end. Not 1: under
EOS the last offset is
+ // usually a transaction control record, which is never delivered to a
consumer, so probing
+ // there is a guaranteed empty poll. The first window answers the large
majority; only the
+ // partitions it cannot answer for pay to widen.
+ private static final long[] PROBE_WINDOWS = {128L, 512L, 2048L};
Review Comment:
You may have had this before but consider adding a shallower rung:
`PROBE_WINDOWS = {8L, 128L, 512L, 2048L}`. The probe fetches the entire
window, but we only need one record from it — so starting smaller cuts traffic.
I know this is a reversal of a comment I had earlier but some local testing
convinced me otherwise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
