Hi, currently, a Kafka Streams application is designed to "run forever" and there is no notion of "End of Batch" -- we have plans to add this though... (cf. https://cwiki.apache.org/confluence/display/KAFKA/KIP-95%3A+Incremental+Batch+Processing+for+Kafka+Streams)
Thus, right now you need to stop your application manually. You would need to observe the application's committed offsets (and lag) using bin/kafka-consumer-groups.sh (the application ID is user as group ID) to monitor the app's progress to see when it is done. Cf. https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Streams+Data+%28Re%29Processing+Scenarios -Matthias On 1/28/17 1:07 PM, Eric Dain wrote: > Hi, > > I'm pretty new to Kafka Streams. I am using Kafka Streams to ingest large > csv file. I need to run some clean-up code after all records in the file > are processed. Is there a way to send "End of Batch" event that is > guaranteed to be processed after all records? If not is there alternative > solution? > > Thanks, > Eric >
signature.asc
Description: OpenPGP digital signature
