So I think I figured it out. For some reason I'm getting a db error,
bad transaction id, which then kills my dataManager object, or I should
say the framework is setting it to null. What does a Bad transaction ID
mean? Thoughts? This happened after I did a LockClean and restart both
the agent and Tomcat. Thanks, Farzad.
ERROR 2011-06-07 11:44:56,365 [Worker thread '90']
(CacheManager.java:621) - Thread[Worker thread '90',5,main]:
invalidateKeys: 1307465096157:
org.apache.manifoldcf.core.cachemanager.CacheManager@13b0c258:
Transaction hash =
{1307465096144=org.apache.manifoldcf.core.cachemanager.CacheManager$CacheTransactionHandle@39a72981}
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad
transaction ID!
at
org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620)
at
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175)
at
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168)
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637)
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191)
at
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76)
at
org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115)
at
org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565)
at
org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275)
at
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564)
FATAL 2011-06-07 11:44:56,583 [Worker thread '32']
(DupFinderConnector.java:155) - DATAMANAGER IS NULL!!!!
ERROR 2011-06-07 11:44:56,599 [Worker thread '90']
(WorkerThread.java:893) - Exception tossed: Bad transaction ID!
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad
transaction ID!
at
org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620)
at
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175)
at
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168)
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637)
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191)
at
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76)
at
org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115)
at
org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565)
at
org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275)
at
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564)
FATAL 2011-06-07 11:44:56,614 [Worker thread '32']
(WorkerThread.java:955) - Error tossed: null
java.lang.NullPointerException
at
org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565)
at
org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275)
at
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564)
INFO 2011-06-07 11:44:56,645 [Worker thread '92']
(DupFinderConnector.java:251) - Attempting to initialize
dataManager(null) and ciConnector(null)
On 6/7/2011 9:20 AM, Farzad Valad wrote:
Lately when I issue an abort on a crawl job (click abort in UI), it
gets stuck, meaning the UI doesn't show any new info on subsequent
refreshes. It just says Aborting, the start time, no end time, shows
# of documents, active, and processed. I restarted Tomcat, but still
stuck in Aborting state. Restarting the Agent process doesn't have
any affect. But now if you kill the agent process and issue lock
clean, then start the Agent Process, it will show an Error in the
Status column, but no end time. Ironically, this time the problem was
a bad transaction id. The last time it was a connection refusal to my
repository. Thoughts?
PS. Previous problem, you were right, dataManager is going null for
some reason, actually debugging for dataManager I ran into this one : )
ERROR 2011-06-07 08:50:01,416 [Worker thread '64']
(CacheManager.java:621) - Thread[Worker thread '64',5,main]:
invalidateKeys: 1307454600471:
org.apache.manifoldcf.core.cachemanager.CacheManager@39d7af3:
Transaction hash = {}
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad
transaction ID!
at
org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620)
at
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175)
at
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168)
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637)
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191)
at
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76)
at
org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115)
at
org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:162)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565)
at
org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275)
at
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564)
ERROR 2011-06-07 08:50:01,510 [Worker thread '64']
(WorkerThread.java:893) - Exception tossed: Bad transaction ID!
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad
transaction ID!
at
org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620)
at
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175)
at
org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168)
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637)
at
org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191)
at
org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76)
at
org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115)
at
org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:162)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565)
at
org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275)
at
org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564)
