VartulAgrawal commented on a change in pull request #1177: SAMZA-2339 : Update
KafkaSystemAdmin to adhere to the checkpoint replication factor
URL: https://github.com/apache/samza/pull/1177#discussion_r331292767
##########
File path:
samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java
##########
@@ -540,6 +540,9 @@ public KafkaStreamSpec toKafkaSpec(StreamSpec spec) {
kafkaSpec =
new KafkaStreamSpec(spec.getId(), spec.getPhysicalName(),
systemName, 1, coordinatorStreamReplicationFactor,
coordinatorStreamProperties);
+ } else if (spec.isCheckpointStream()) {
+ kafkaSpec =
KafkaStreamSpec.fromSpec(StreamSpec.createCheckpointStreamSpec(spec.getPhysicalName(),
systemName))
+ .copyWithReplicationFactor(Integer.parseInt(new
KafkaConfig(config).getCheckpointReplicationFactor().get()));
Review comment:
The KafkaCheckpointManagerFactory is only created after the KafkaSystemAdmin
is initialised so by the time KafkaCheckpointManagerFactory is called the
replicationFactor is already reset to 2
(https://github.com/apache/samza/blob/master/samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java#L552
which is default for any general topic) and hence it needs to be passed to the
spec here. Tested this in our current setup too :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
