Here is the scenario I have a client using Kubernetes and we run a single pipeline in a pod.. We scale up/down bases on queue pressure..
I’m observing on a consistent basis that the 1, first time the pipelines start(one pipeline), the pipeline will pull 2 documents off of the input queue.. 2. The pipeline is shutdown with SIGTERM.. It completes the first document and returns the CAS. It also returns the second CAS it had pulled. 3. A new pipeline is started.. 4. It pulls 2 documents off the input queue. 5. The pipeline is shutdown with a SIGTERM. It completes the document and returns the CAS.. BUT this time it DOES NOT return the second CAS. This happens consistently. So, I’m loosing documents. I cannot for the life of life of me figure out why the new pipeline is not returning the CAS.. This is related to another email I sent last week. But I assumed it was the (what appears to be) extra consumer.. We have 3 consumers, 1 for meta, 1 for messages and apparently there is another one that I don’t know what it’s purpose is but is not related to the consumer pulling multiple records off the queue. So, is there a way I can get the consumer to only pull 1 document off the queue? If not, is the second CAS not being returned a bug? Im running 2.10.3, uimaj-as.. We use ddspring to start things up. I even set the pipeline numberOfCases to 1 thinking it may only pull a single doc off the input queue, but it still pulled 2 Daniel Cosio dcco...@gmail.com
