[
https://issues.apache.org/jira/browse/SAMZA-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yi Pan resolved SAMZA-2431.
---------------------------
Resolution: Fixed
The PR has been merged. Closing now.
> Fix the checkpoint and change-log topic-creation workflow.
> ----------------------------------------------------------
>
> Key: SAMZA-2431
> URL: https://issues.apache.org/jira/browse/SAMZA-2431
> Project: Samza
> Issue Type: Bug
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Symptom: Checkpoint and changelog kafka topics are expected to be created for
> a samza job with cleanup.policy set to compact. This ensures that all the
> data in the metadata topics are log-compacted. However, we observed that
> checkpoint & changelog topics for hundreds of production samza jobs were
> created with cleanup.policy set to delete leading to strong-possibility of
> message-l0ss.
> Cause:
> - *Checkpoint:* The control-flow in KafkaStreamSpec to build checkpoint spec
> swallows the essential kafka-topic configuration rather passes empty
> configuration bag to kafka-broker(via kafka-rest). This issue was introduced
> in SAMZA-2339.
> - *Changelog:* There change-log topic configurations are incorrectly
> generated when the RocksDB store TTL is set to -1 by the user.
> Changes: Change the topic-creation control-flow for above metadata topics to
> generate correct topic-configurations.
> Tests: Added unit tests to validate the expected topic configuration bag.
> API Changes: None
> Upgrade Instructions: None
> Usage Instructions: None
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
