The BEINGDELETED document state is insufficient to allow proper error recovery
from a search engine that's down
---------------------------------------------------------------------------------------------------------------
Key: CONNECTORS-145
URL: https://issues.apache.org/jira/browse/CONNECTORS-145
Project: ManifoldCF
Issue Type: Bug
Reporter: Karl Wright
The expire stuffer thread puts documents from both the PENDING and
PENDINGPURGATORY states into the BEINGDELETED state. But this state is
insufficient to handle proper error recovery in the case of a down search
engine blocking expiration. What should happen is that the error from the
failed expiration attempt should move the candidate document back into its
original state for a later expiration attempt, but right now the Expire Thread
simply blocks and retries, essentially using up an expire thread for the
duration of the outage.
In a time when there was only one output connection, this was acceptable logic,
but since there are multiple such connections possible now, it is a potential
liability.
The way forward is to introduce a new document state, BEINGDELETEDPURGATORY,
and insure that the mapping between PENDING->BEINGDELETED and
PENDINGPURGATORY->BEINGDELETEDPURGATORY is preserved. Then, logic must change
in the Expire Thread to perform the appropriate error recovery.
The other thread family that has this problem is the Document Delete Thread
family. These threads will require some thought to fix because there is
currently no deletion scheduling field in the jobqueue database table, and yet
we'd need one if we were going to fix this problem in an appropriate manner.
However, we can probably reuse the checktime field for this purpose if we are
clever.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
