akhil-sabu-ibm opened a new pull request, #22324: URL: https://github.com/apache/kafka/pull/22324
This PR improves the retry logic in `KafkaStatusBackingStore` to address the TODO comment at line 283. ## Problem Previously, the code retried indefinitely without backoff when encountering `RetriableException`, which could: - Exhaust system resources during prolonged outages - Overwhelm Kafka brokers with rapid retry attempts - Lack visibility into retry behavior ## Solution Implemented graceful retry mechanism with: - **Exponential backoff**: 300ms → 600ms → 1.2s → 2.4s → ... → 60s (max) - **Retry limit**: Maximum 10 attempts before giving up - **Enhanced logging**: Tracks retry attempts, backoff times, and final outcomes - **Resource protection**: Prevents infinite loops while maintaining resilience ## Changes - Added retry configuration constants (`MAX_RETRY_ATTEMPTS=10`, `INITIAL_RETRY_BACKOFF_MS=300`, `MAX_RETRY_BACKOFF_MS=60000`) - Implemented `sendWithRetry()` methods with exponential backoff for both topic status and general status updates - Implemented `calculateBackoff()` using formula: `min(INITIAL_BACKOFF * 2^attempt, MAX_BACKOFF)` - Updated `sendTopicStatus()` and `send()` methods to use new retry logic - Added comprehensive JavaDoc documentation - Preserved existing safe-write checks and generation validation ## Testing - ✅ All 24 existing tests pass without modification - ✅ Compilation successful with Java 21 - ✅ Checkstyle and SpotBugs checks passed - ✅ Maintains backward compatibility - ✅ No breaking changes ## Benefits - Prevents resource exhaustion during Kafka broker outages - Reduces load on brokers with exponential backoff - Improves observability with detailed retry logging - Aligns with best practices used in other Kafka Connect components (e.g., `RetryUtil`, `DistributedHerder`) JIRA: https://issues.apache.org/jira/browse/KAFKA-20599 This is an improvement to system reliability with no breaking changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
