Re: Data Manager Null

Tue, 07 Jun 2011 13:39:11 -0700

I think I found the problem. I should be tearing down the dataManager and recreating it between clear and set thread context calls, because it has a thread context. I'm not doing that. I guess I did learn something reading : ) let me know if you believe otherwise. Also do you think this is why the bad transaction id is happening? Thanks!
IDBInterface databaseHandle = DBInterfaceFactory.make(currentContext, ManifoldCF.getMasterDatabaseName(), ManifoldCF.getMasterDatabaseUsername(), ManifoldCF.getMasterDatabasePassword()); 
            dataManager = new DataManager(currentContext, databaseHandle);



On 6/7/2011 12:42 PM, Farzad Valad wrote:
So I think I figured it out. For some reason I'm getting a db error, bad transaction id, which then kills my dataManager object, or I should say the framework is setting it to null. What does a Bad transaction ID mean? Thoughts? This happened after I did a LockClean and restart both the agent and Tomcat. Thanks, Farzad.
ERROR 2011-06-07 11:44:56,365 [Worker thread '90'] (CacheManager.java:621) - Thread[Worker thread '90',5,main]: invalidateKeys: 1307465096157: org.apache.manifoldcf.core.cachemanager.CacheManager@13b0c258: Transaction hash = {1307465096144=org.apache.manifoldcf.core.cachemanager.CacheManager$CacheTransactionHandle@39a72981} org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad transaction ID! at org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175) at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191) at org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76) at org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115) at org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) FATAL 2011-06-07 11:44:56,583 [Worker thread '32'] (DupFinderConnector.java:155) - DATAMANAGER IS NULL!!!! ERROR 2011-06-07 11:44:56,599 [Worker thread '90'] (WorkerThread.java:893) - Exception tossed: Bad transaction ID! org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad transaction ID! at org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175) at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191) at org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76) at org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115) at org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) FATAL 2011-06-07 11:44:56,614 [Worker thread '32'] (WorkerThread.java:955) - Error tossed: null 
java.lang.NullPointerException
at org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) INFO 2011-06-07 11:44:56,645 [Worker thread '92'] (DupFinderConnector.java:251) - Attempting to initialize dataManager(null) and ciConnector(null) 


On 6/7/2011 9:20 AM, Farzad Valad wrote:
Lately when I issue an abort on a crawl job (click abort in UI), it gets stuck, meaning the UI doesn't show any new info on subsequent refreshes. It just says Aborting, the start time, no end time, shows # of documents, active, and processed. I restarted Tomcat, but still stuck in Aborting state. Restarting the Agent process doesn't have any affect. But now if you kill the agent process and issue lock clean, then start the Agent Process, it will show an Error in the Status column, but no end time. Ironically, this time the problem was a bad transaction id. The last time it was a connection refusal to my repository. Thoughts?
PS. Previous problem, you were right, dataManager is going null for some reason, actually debugging for dataManager I ran into this one : )
ERROR 2011-06-07 08:50:01,416 [Worker thread '64'] (CacheManager.java:621) - Thread[Worker thread '64',5,main]: invalidateKeys: 1307454600471: org.apache.manifoldcf.core.cachemanager.CacheManager@39d7af3: Transaction hash = {} org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad transaction ID! at org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175) at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191) at org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76) at org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115) at org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:162) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) ERROR 2011-06-07 08:50:01,510 [Worker thread '64'] (WorkerThread.java:893) - Exception tossed: Bad transaction ID! org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad transaction ID! at org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620) at org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175) at org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637) at org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191) at org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76) at org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115) at org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:162) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) at org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564)

Reply via email to