Hello Dan, It seems you are hitting a known issue that KIP-91 is trying to fix (it is a general issue of producer itself): https://cwiki.apache.org/confluence/display/KAFKA/KIP-91+Provide+Intuitive+User+Timeouts+in+The+Producer
It means that the records has never been sent out before expiring since the metadata for finding the leader of "my-table-changelog-18" is never known, it is likely due to your broker is not available and hence metadata refresh never succeed. Hence, retries config does not help here since it is never sent actually. Before KIP-91 gets merged in, you will have to increase your request.timeout.ms (btw it is already as high as 30 secs, do you have a broker offline for about that long period?) to larger value so that the producer can tolerate longer time that metadata is not available for sending. Guozhang On Tue, Jan 23, 2018 at 5:27 PM, dan bress <danbr...@gmail.com> wrote: > I'm seeing this timeout in my Kafka Streams application logs: > > Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 7 > record(s) for my-table-changelog-18: 30011 ms has passed since last append > > What kind of retries to I get by default publishing to changelogs? I have > these producer configs set: > > 2018-01-23 21:14:31,197 INF [main] ProducerConfig ProducerConfig values: > acks = all > retries = 30 > retry.backoff.ms = 1000 > request.timeout.ms = 30000 > > Does this mean I am getting 30 retries when trying to publish to > changelogs? Or do I need to set another config(StreamsConfig) to get more > retries for changelog publishing? > > Thanks! > Dan > -- -- Guozhang
