Mehari Beyene created KAFKA-18463:
-------------------------------------
Summary: Backward compatibility issue with the configs
log.message.timestamp.after.max.ms and log.message.timestamp.before.max.ms
Key: KAFKA-18463
URL: https://issues.apache.org/jira/browse/KAFKA-18463
Project: Kafka
Issue Type: Improvement
Affects Versions: 3.8.1, 3.7.2, 3.9.0, 3.7.1, 3.8.0, 3.6.2, 3.6.1, 3.7.0,
3.6.0, 3.6.3, 3.9.1, 3.8.2
Reporter: Mehari Beyene
With [KIP-937|https://cwiki.apache.org/confluence/display/KAFKA/KIP-937] we
have introduced two new configs,
`log.message.timestamp.before.max.ms` and `log.message.timestamp.after.max.ms`
that are used to validate the timestamp of messages sent by producer. The
rational for adding these configs is discussed in the KIP but the main
motivation is messages with invalid timestamp interfere with log rotation and
log cleaning, because these functions depends on comparing the current
timestamp of the server with the message timestamp to make the decision weather
a log files should be rotated or evicted.
However there is backward compatibility issue.
For example if the cluster is upgraded to a 3.6.0 and above versions from a
version that did not have these config options and contains message with
invalid timestamp like nanosecond instead of a millisecond with future
timestamp like `{{{}2446236110000{}}}` -> 22 years in the future, both the log
cleaner and log rotation logic will still be impaired even if we have
`log.message.timestamp.after.max.ms` configured with the proposed default value
of {{{}`86400000{}}}` for version 4.0 and above. Reference [KIP
1030|https://cwiki.apache.org/confluence/display/KAFKA/KIP-1030]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
