xinyuiscool commented on a change in pull request #993: SAMZA-2164: Close the
consumer after reading checkpoints
URL: https://github.com/apache/samza/pull/993#discussion_r275086566
##########
File path:
samza-kafka/src/main/scala/org/apache/samza/checkpoint/kafka/KafkaCheckpointManager.scala
##########
@@ -128,11 +128,11 @@ class KafkaCheckpointManager(checkpointSpec:
KafkaStreamSpec,
info(s"Reading checkpoint for taskName $taskName")
if (taskNamesToCheckpoints == null) {
- debug("Reading checkpoints for the first time")
+ info("Reading checkpoints for the first time")
taskNamesToCheckpoints = readCheckpoints()
- } else {
- debug("Updating existing checkpoint mappings")
- taskNamesToCheckpoints ++= readCheckpoints()
+ // Stop the system consumer since we only need to read checkpoints once
Review comment:
No, i don't think that update is needed at all. It's weird after the first
pulling we are still trying to update. It doesn't make it better if there is
any problem in the first read (some task also has wrong checkpoints).
I feel it might be useful to start a little bit early to just give a bit
more time to let the proxy thread running. It's not used anywhere else.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
