Postgresql load test job delete document cleanup fails sometimes and leaves
orphaned documents
----------------------------------------------------------------------------------------------
Key: CONNECTORS-279
URL: https://issues.apache.org/jira/browse/CONNECTORS-279
Project: ManifoldCF
Issue Type: Bug
Components: Framework crawler agent
Affects Versions: ManifoldCF 0.4
Reporter: Karl Wright
Assignee: Karl Wright
Fix For: ManifoldCF 0.4
Running the postgresql load test on my laptop, I was surprised when the test
did not finish. The UI indicated that the job was being deleted, but there
were 49,000 documents and that number was not moving. Further inspection
yielded the following:
- Job was in the "DELETING" state
- Documents were in the "BEINGDELETED" state
- No activity of any kind ongoing
The log had no errors.
It was impossible to get a thread dump, but a cursory inspection of the code
indicated that either the documents were being marked as "BEINGDELETED" but
were not actually being placed on the in-memory queue, or the delete threads
were picking up the documents and somehow avoiding marking them as being
processed.
Also, probably unrelated, the Document Status report listed these documents as
having a status of "Being removed" and a state of "Unknown". The "Unknown"
should have been a "Deleting". Since the extended WHEN... ELSE clause has a
reasonable condition for the "Deleting" answer, it's hard to see how this could
have occurred either.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
