Christo Lolov created KAFKA-15195:
-------------------------------------
Summary: Regenerate segment-aligned producer snapshots when
upgrading to a Kafka version supporting Tiered Storage
Key: KAFKA-15195
URL: https://issues.apache.org/jira/browse/KAFKA-15195
Project: Kafka
Issue Type: Sub-task
Affects Versions: 3.6.0
Reporter: Christo Lolov
Assignee: Christo Lolov
As mentioned in KIP-405: Kafka Tiered Storage#Upgrade a customer wishing to
upgrade from a Kafka version < 2.8.0 to 3.6 and turn Tiered Storage on will
have to wait for retention to clean up segments without an associated producer
snapshot.
However, in our experience, customers of Kafka expect to be able to immediately
enable tiering on a topic once their cluster upgrade is complete. Once they do
this, however, they start seeing NPEs and no data is uploaded to Tiered Storage
(https://github.com/apache/kafka/blob/9e50f7cdd37f923cfef4711cf11c1c5271a0a6c7/storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/LogSegmentData.java#L61).
To achieve this, we propose changing Kafka to retroactively create producer
snapshot files on upload whenever a segment is due to be archived and lacks one.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
