[
https://issues.apache.org/jira/browse/KAFKA-7897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson updated KAFKA-7897:
-----------------------------------
Description:
Message format downgrades are not supported, but they generally work as long as
broker/clients at least can continue to parse both message formats. After a
downgrade, the truncation logic should revert to using the high watermark, but
currently we use the existence of any cached epoch as the sole prerequisite in
order to leverage OffsetsForLeaderEpoch. This has the effect of causing a
massive truncation after startup which causes re-replication.
I think our options to fix this are to either 1) clear the cache when we notice
a downgrade, or 2) forbid downgrades and raise an error.
was:
Message format downgrades are not supported, but they generally work as long as
broker/clients at least can continue to parse both message formats. After a
downgrade, the truncation logic should revert to using the high watermark, but
currently we use the existence of any cached epoch as the requirement in order
to leverage OffsetsForLeaderEpoch. This has the effect of causing a massive
truncation after startup which causes re-replication.
I think our options to fix this are to either 1) clear the cache when we notice
a downgrade, or 2) forbid downgrades and raise an error.
> Invalid use of epoch cache following message format downgrade
> -------------------------------------------------------------
>
> Key: KAFKA-7897
> URL: https://issues.apache.org/jira/browse/KAFKA-7897
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Priority: Major
>
> Message format downgrades are not supported, but they generally work as long
> as broker/clients at least can continue to parse both message formats. After
> a downgrade, the truncation logic should revert to using the high watermark,
> but currently we use the existence of any cached epoch as the sole
> prerequisite in order to leverage OffsetsForLeaderEpoch. This has the effect
> of causing a massive truncation after startup which causes re-replication.
> I think our options to fix this are to either 1) clear the cache when we
> notice a downgrade, or 2) forbid downgrades and raise an error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
