Hi, I am pretty new to Zoo keeper and book keeper.
Right now, I am implementing a distributed data processing pipeline, which includes multiple stages. The input is a stream of product entries. I like to use Book keeper to do write ahead logging for the input to the pipeline so that if some process dies, the pipeline can recover from the server failure simply by reading the unprocessed log. What I want is to first write each input product info into the log using book keeper and mark or delete the product info once the pipeline finishes processing the product. I am not sure how to apply book keeper for this scenario, more specifically, how to mark or delete the log after the product is out of the pipeline. Also, I wonder if I can embed the book keeper server inside my application to reduce the operational overhead. Your help is highly appreciate. Thanks, John
