jberstler commented on issue #234: requeue messages to kafka if subject is 
throttled
URL: 
https://github.com/apache/incubator-openwhisk-package-kafka/issues/234#issuecomment-362605056
 
 
   Like @csantanapr, I find it a little hard to believe we should expect 
customers to set up a separate topic for us, as well as allow us write access 
to it, in order to handle the case where OpenWhisk cannot (or will not) handle 
the rate at which they are producing messages. This is not to mention asking 
them to configure their triggers to make use of this new topic, which adds even 
more configuration burden and potential confusion when creating the trigger.
   
   Like @abaruni, I think the matter could be much more simply handled by 
allowing a longer retry period - possibly configurable by the trigger creator. 
Because the message contents are kept in memory, this retry period could, at 
least in theory, extend even beyond the customer's own Kafka retention policy - 
so long as the provider instance didn't lose the contents of its memory (say, 
through a reboot). I'm not sure I see a benefit to adjusting the exponential 
nature of the retry behavior. Once you start waiting > 60 seconds between 
retries, I don't think the exact value of how long you wait is really all that 
significant.
   
   I also think it is reasonable that if the customer wants to ensure that no 
messages are lost due to throttling, that they increase their cluster's message 
retention period in addition to configuring the new retry timeout parameter on 
their trigger.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to