windhamwong edited a comment on issue #3048: Kafka spout URL: https://github.com/apache/incubator-heron/issues/3048#issuecomment-434170689 I currently have the Kafka spout code working, however, it falls after some time randomly (around 2 - 7 days). It shows initialized but nothing received from the Kafka broker. There is no sign or error showing its falling. Only thing I can notice is, the Kafka Spout reinitialize when it says Restoring instance after some time. ``` [2018-10-30 04:06:41 +0000] [INFO] st_heron_instance.py: Restoring instance state to checkpoint 1540822768709483564-1540872269 [2018-10-30 04:06:41 +0000] [INFO] st_heron_instance.py: Instance restore state deserialized [2018-10-30 04:06:42 +0000] [INFO] st_heron_instance.py: Received start stateful processing for 1540822768709483564-1540872269 [2018-10-30 04:06:42 +0000] [INFO] st_heron_instance.py: Starting bolt/spout instance now... [2018-10-30 04:06:42 +0000] [INFO] base_instance.py: [+] Initializing InputKafkaLogSpout... ``` I also tried to use 3 or 1 Kafka Spouts as parallel Spout, but still not fixing the issue. However, one thing I can confirm is that, 3 parallel spouts setup doesn't duplicate message emit.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
