On Tue, May 28, 2024 at 3:40 PM Daniel Cosio <dcco...@gmail.com> wrote:
> I’m seeing a couple of issues with the pipelines that cause messages(and > CAS’s) to be lost. > > The pipeline is prefetching 2 messages off the input queue. If the > pipeline receives a SIGTERM is will shut down. In this case, the document > has finished but we loose the second document that was prefetched.. > I never see the second prefetched document back on the input queue, so Im > assuming it must have be ack’d(which doesn’t seem that this should > happen)..I also seem to loose both CAS’s from the reader because > The callback to the reader to tell it the CAS can be released is never > called… > > > This issue surfaced when moving our implementation to Kubernetes.. The > shutdown is happening when a pod is stopped. I have a process that will > requeue the last DOCID which is ok, but I have no way of recovering the > second prefetched document.. > > I use UimaJ and Uima-AS 2.10.2. We cannot upgrade to 3.x yet. > > When I was running on 2.6 there were 2 consumers connected to the input > queue.. With 2.10.2 there are 3 and Im suspecting the extra consumer is the > one grabbing the second message. > > So, a few questions. > > 1. Is there a way via config to stop the pipeline from prefetching the > second document > 2. Is there a way to (via a shutdown hook) to let the pipeline return > CAS’s to the reader if it recieves a SIGTERM. > 3. Is there a way go back to the 2 consumers? > > Any insight would be appreciated > Hi Daniel, What you describe is very unfortunate. I did a search on the mailing lists and found this issue that talks about Kubernets and SIGTERM changes to the shutdown hooks: https://issues.apache.org/jira/browse/UIMA-5909 You might want to look at what was done to close it and try reverting that change to see if it fixes your issue. The three consumers are explained in this email: https://lists.apache.org/thread/sm3g4yszsnygj3gxsxj16b6q6nozwcxb It doesn't seem the jump from 2 consumers to 3 might be related to what you're observing. Hope this helps, P > > > Daniel Cosio > dcco...@gmail.com > > > >
