chenboat opened a new pull request #4618: Make Kafka offset out of range as an transient exception. URL: https://github.com/apache/incubator-pinot/pull/4618 During LLC migration in Uber, we often found LLC consumer often stopped consumption because it is assigned a bad Kafka broker (and resulted in OFFSET_OUT_OF_RANGE error). The current way to assigning OFFSET_OUT_OF_RANGE as permanent error causing the realtime consumption to stop -- until a background fixer thread to revive the consumption in ~10mins. In a company with large number of Kafka brokers, faulty brokers occur pretty frequently. A stop of realtime ingestion for 10 mins is not acceptable due to high SLA. We rather mark the error as a transient one and let the retry to kick in to select a different brokers. It works well after the error code reassignments.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
