markusthoemmes commented on a change in pull request #3072: Enhance kafka
message provider
URL:
https://github.com/apache/incubator-openwhisk/pull/3072#discussion_r160156426
##########
File path:
common/scala/src/main/scala/whisk/connector/kafka/KafkaProducerConnector.scala
##########
@@ -66,12 +70,33 @@ class KafkaProducerConnector(kafkahosts: String,
case Failure(t) =>
logging.error(this, s"sending message on topic '$topic' failed:
${t.getMessage}")
} recoverWith {
- case t: NotLeaderForPartitionException =>
- if (retry > 0) {
- logging.error(this, s"NotLeaderForPartitionException is retryable,
remain $retry retry")
- Thread.sleep(100)
- send(topic, msg, retry - 1)
- } else produced.future
+
+ /**
+ * Kafka Error handling strategy
+ *
+ * 1. Error we know of can be handled by retry
+ * - InvalidMetadataException: An exception that may indicate the
client's metadata is out of date,
+ * includes NetworkException,
KafkaStorageException, LeaderNotAvailableException,
+ * NotLeaderForPartitionException,
UnknownTopicOrPartitionException.
+ *
+ * 2. Errors we don't know of just result in a restart of the
consumer/producer
+ *
+ */
+ case e: KafkaException =>
Review comment:
Can we use `RetriableException` and exclude the ones we deem harmful?
```scala
case _: NotEnoughReplicasAfterAppendException =>
recreateProducer()
produced.future
case _: RetriableException =>
...
case _ =>
recreateProducer()
produced.future
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
