KKcorps commented on code in PR #20137:
URL: https://github.com/apache/pinot/pull/20137#discussion_r4586111212
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java:
##########
@@ -285,6 +287,15 @@ public void deleteSegmentFile() {
private volatile long _numBytesDropped = 0;
private volatile int _consecutiveErrorCount = 0;
private volatile boolean _firstDecodeErrorLogged = false;
+ // Decode retry state, only accessed by the consumer thread
Review Comment:
Nit: catch up in `CONSUMING_TO_ONLINE` runs this on the Helix thread. It's
safe because the consumer thread has stopped by then, but the comment could say
so.
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/writer/StatelessRealtimeSegmentWriter.java:
##########
@@ -272,6 +273,13 @@ public void run() {
_consumer.start(_startOffset);
_logger.info("Created new consumer thread {} for {}", _consumerThread,
this);
_currentOffset = _startOffset;
+ // Decode retry state of the current message, and the ongoing decode
outage as in RealtimeSegmentDataManager
+ int numDecodeRetries = 0;
+ long decodeRetryBackoffMs = 0;
+ boolean inDecodeOutage = false;
Review Comment:
This is a second copy of the outage logic in `shouldRetryDecode`, and the
two have already drifted, since this one has no once a minute WARN and no
segment error. Could we move it into one small class that both use?
##########
pinot-spi/src/main/java/org/apache/pinot/spi/stream/StreamConfigProperties.java:
##########
@@ -127,6 +127,17 @@ private StreamConfigProperties() {
/// Config used to indicate whether the topic is a temporary topic for
offset auto reset backfilling
public static final String BACKFILL_TOPIC =
"realtime.segment.isBackfillTopic";
+ /// Cap, in milliseconds, of the backoff before a message whose decoder
threw [RetriableDecodeException] is fetched
+ /// again. The backoff starts at
[StreamConfig#INITIAL_DECODE_RETRY_BACKOFF_MILLIS] and doubles up to the cap.
+ /// Defaults to [StreamConfig#DEFAULT_DECODE_RETRY_MAX_BACKOFF_MILLIS] (10
seconds).
+ public static final String DECODE_RETRY_MAX_BACKOFF_MILLIS =
"decodeRetryMaxBackoffMillis";
+
+ /// How long, in milliseconds, retriable decode failures are retried during
an outage of a decoder dependency. An
+ /// outage lasts while a message is being retried or retriable failures keep
happening less than this apart; once it
+ /// has lasted this long, its failures are handled like any other decode
failure. No limit by default; `0` disables
+ /// retries.
+ public static final String DECODE_RETRY_MAX_DURATION_MILLIS =
"decodeRetryMaxDurationMillis";
Review Comment:
The outage state lives on the segment, so a segment that ends on the time
limit while stuck gives the next one a fresh window, and so does a pause,
restart or rebalance. Could the doc say the limit is per segment, so nobody
sets it expecting a hard cap?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]