It sounds like you are on the right track for fixing all of these problems.
Karl On Tue, Jun 7, 2011 at 4:38 PM, Farzad Valad <[email protected]> wrote: > I think I found the problem. I should be tearing down the dataManager and > recreating it between clear and set thread context calls, because it has a > thread context. I'm not doing that. I guess I did learn something reading > : ) let me know if you believe otherwise. Also do you think this is why the > bad transaction id is happening? Thanks! > > IDBInterface databaseHandle = > DBInterfaceFactory.make(currentContext, ManifoldCF.getMasterDatabaseName(), > ManifoldCF.getMasterDatabaseUsername(), > ManifoldCF.getMasterDatabasePassword()); > dataManager = new DataManager(currentContext, databaseHandle); > > > > On 6/7/2011 12:42 PM, Farzad Valad wrote: >> >> So I think I figured it out. For some reason I'm getting a db error, bad >> transaction id, which then kills my dataManager object, or I should say the >> framework is setting it to null. What does a Bad transaction ID mean? >> Thoughts? This happened after I did a LockClean and restart both the agent >> and Tomcat. Thanks, Farzad. >> >> ERROR 2011-06-07 11:44:56,365 [Worker thread '90'] (CacheManager.java:621) >> - Thread[Worker thread '90',5,main]: invalidateKeys: 1307465096157: >> org.apache.manifoldcf.core.cachemanager.CacheManager@13b0c258: Transaction >> hash = >> {1307465096144=org.apache.manifoldcf.core.cachemanager.CacheManager$CacheTransactionHandle@39a72981} >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad transaction >> ID! >> at >> org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620) >> at >> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175) >> at >> org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168) >> at >> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637) >> at >> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191) >> at >> org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76) >> at >> org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115) >> at >> org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) >> at >> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) >> at >> org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) >> at >> org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) >> at >> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) >> FATAL 2011-06-07 11:44:56,583 [Worker thread '32'] >> (DupFinderConnector.java:155) - DATAMANAGER IS NULL!!!! >> ERROR 2011-06-07 11:44:56,599 [Worker thread '90'] (WorkerThread.java:893) >> - Exception tossed: Bad transaction ID! >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad transaction >> ID! >> at >> org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620) >> at >> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175) >> at >> org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168) >> at >> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637) >> at >> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191) >> at >> org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76) >> at >> org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115) >> at >> org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) >> at >> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) >> at >> org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) >> at >> org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) >> at >> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) >> FATAL 2011-06-07 11:44:56,614 [Worker thread '32'] (WorkerThread.java:955) >> - Error tossed: null >> java.lang.NullPointerException >> at >> org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:158) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) >> at >> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) >> at >> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) >> at >> org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) >> at >> org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) >> at >> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) >> INFO 2011-06-07 11:44:56,645 [Worker thread '92'] >> (DupFinderConnector.java:251) - Attempting to initialize dataManager(null) >> and ciConnector(null) >> >> >> On 6/7/2011 9:20 AM, Farzad Valad wrote: >>> >>> Lately when I issue an abort on a crawl job (click abort in UI), it gets >>> stuck, meaning the UI doesn't show any new info on subsequent refreshes. It >>> just says Aborting, the start time, no end time, shows # of documents, >>> active, and processed. I restarted Tomcat, but still stuck in Aborting >>> state. Restarting the Agent process doesn't have any affect. But now if >>> you kill the agent process and issue lock clean, then start the Agent >>> Process, it will show an Error in the Status column, but no end time. >>> Ironically, this time the problem was a bad transaction id. The last time >>> it was a connection refusal to my repository. Thoughts? >>> >>> PS. Previous problem, you were right, dataManager is going null for some >>> reason, actually debugging for dataManager I ran into this one : ) >>> >>> ERROR 2011-06-07 08:50:01,416 [Worker thread '64'] >>> (CacheManager.java:621) - Thread[Worker thread '64',5,main]: invalidateKeys: >>> 1307454600471: org.apache.manifoldcf.core.cachemanager.CacheManager@39d7af3: >>> Transaction hash = {} >>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad >>> transaction ID! >>> at >>> org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620) >>> at >>> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175) >>> at >>> org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168) >>> at >>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637) >>> at >>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191) >>> at >>> org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76) >>> at >>> org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115) >>> at >>> org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:162) >>> at >>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) >>> at >>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) >>> at >>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) >>> at >>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) >>> at >>> org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) >>> at >>> org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) >>> at >>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) >>> ERROR 2011-06-07 08:50:01,510 [Worker thread '64'] >>> (WorkerThread.java:893) - Exception tossed: Bad transaction ID! >>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Bad >>> transaction ID! >>> at >>> org.apache.manifoldcf.core.cachemanager.CacheManager.invalidateKeys(CacheManager.java:620) >>> at >>> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:175) >>> at >>> org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:168) >>> at >>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:637) >>> at >>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:191) >>> at >>> org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:76) >>> at >>> org.apache.manifoldcf.agents.output.dupfinder.DataManager.insertData(DataManager.java:115) >>> at >>> org.apache.manifoldcf.agents.output.dupfinder.DupFinderConnector.addOrReplaceDocument(DupFinderConnector.java:162) >>> at >>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.addOrReplaceDocument(IncrementalIngester.java:1433) >>> at >>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.performIngestion(IncrementalIngester.java:418) >>> at >>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:313) >>> at >>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocument(WorkerThread.java:1565) >>> at >>> org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:275) >>> at >>> org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector.processDocuments(BaseRepositoryConnector.java:423) >>> at >>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:564) >>> >> > >
