Michael Axiak created KAFKA-8395:
------------------------------------
Summary: Add an ability to backup log segment files on truncation
Key: KAFKA-8395
URL: https://issues.apache.org/jira/browse/KAFKA-8395
Project: Kafka
Issue Type: New Feature
Components: core
Affects Versions: 2.2.0
Reporter: Michael Axiak
Fix For: 2.2.1
At HubSpot, we believe we hit a combination of bugs [1] [2], which may have
caused us to lose data. In this scenario, as part of metadata conflict
resolution a slowly starting up broker recovered an offset of zero and
truncated segment files.
As part of a belt-and-suspenders approach to reducing this risk in the future,
I propose adding the ability to rename/backup these files and allowing kafka to
move on. Note that this breaks the ordering guarantees, but allows one to
recover the data and decide later how to approach it.
This feature should be turned off by default but enabled with a configuration
option.
(A pull request is following soon on Github)
1: https://issues.apache.org/jira/browse/KAFKA-2178
2: https://issues.apache.org/jira/browse/KAFKA-1120
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
