The service interruption warning is just acknowledgement that you paused or aborted the job. It means nothing.
The warning about long-running queries means that some queries were probably blocked for a fairly long time. This can happen because ManifoldCF periodically analyzes its tables and/or reindexes them. It's usually not a problem except if one particular query appears again and again and seems to have a bad plan, in which case we'd want to know about it so we can fix it. Karl On Tue, Jun 5, 2012 at 5:04 AM, Marcin Goss <[email protected]> wrote: > Ok, after applying your patch, it seems that the problem with duplicate > values is solved. Nice! > However, there are two new warnings that I observed in log: > > 1. WARN 2012-06-05 09:58:03,822 (Worker thread '33') - Pre-ingest service > interruption reported for job 1338817906566 connection 'Wiki': Job no longer > active > > This one is always in series with different numbers of threads. It seems that > I'm getting it after pausing a job. Restarting the job does not help. The > only way to get rid of those warnings is to abort a job and then start it > again. > > 2. WARN 2012-06-05 10:16:52,967 (Worker thread '5') - Found a long-running > query (62959 ms): [UPDATE hopcount SET deathmark=?,distance=? WHERE id > IN(SELECT ownerid FROM hopdeletedeps t0 WHERE t0.jobid=? AND t0.childidhash=? > AND EXISTS(SELECT 'x' FROM intrinsiclink t1 WHERE t1.jobid=t0.jobid AND > t1.linktype=t0.linktype AND t1.parentidhash=t0.parentidhash AND > t1.childidhash=t0.childidhash AND t1.isnew=?))] > WARN 2012-06-05 10:16:52,968 (Worker thread '5') - Parameter 0: 'D' > WARN 2012-06-05 10:16:52,968 (Worker thread '5') - Parameter 1: '-1' > WARN 2012-06-05 10:16:52,968 (Worker thread '5') - Parameter 2: > '1338817906566' > WARN 2012-06-05 10:16:52,968 (Worker thread '5') - Parameter 3: > '9C9DE99F3162AC861D4B92764B9F3CC47F120704' > WARN 2012-06-05 10:16:52,968 (Worker thread '5') - Parameter 4: 'B' > > This one also goes in series. Usually > 15 threads reporting the same warning > one after another. I'm not really sure when it happens though... > > Marcin > > -----Original Message----- > From: Karl Wright [mailto:[email protected]] > Sent: Monday, June 04, 2012 3:03 PM > To: [email protected] > Subject: Re: Derby DB up-to-date? > > Also, for what it is worth, we're planning to release 0.6 sometime around > June 30, provided all the various tickets are completed as expected. > Karl > > On Mon, Jun 4, 2012 at 8:17 AM, Karl Wright <[email protected]> wrote: >> Generally we don't release binaries except as a point release. That >> takes at least a four-day process. I can, of course, build the >> binaries here for you but then I'd have no means of transfering them >> to you. >> >> Can you describe the problems you are having building? I suspect you >> need to download the -lib package and put that in place first. The >> steps are: >> >> (1) download the -src package >> (2) download the -lib package >> (3) unpack them both and place the contents of the -lib package into >> the "lib" directory under the root directory of the -src package >> (where LICENSE.txt is found) >> (4) type "ant build". >> >> There are clear directions in the README.txt file for each of the two >> packages, if you need further help. >> >> Thanks, >> Karl >> >> >> On Mon, Jun 4, 2012 at 8:10 AM, Marcin Goss <[email protected]> wrote: >>> Thanks. Any chance of getting binaries with this fix? I have some problems >>> building everything with the patch applied. >>> >>> -----Original Message----- >>> From: Karl Wright [mailto:[email protected]] >>> Sent: Monday, June 04, 2012 2:05 PM >>> To: [email protected] >>> Subject: Re: Derby DB up-to-date? >>> >>> I committed the pertinent change (without debugging output) to trunk as >>> well. So if you want to upgrade to 0.6-dev, you will get the same fix. >>> >>> Karl >>> >>> >>> On Mon, Jun 4, 2012 at 6:48 AM, Karl Wright <[email protected]> wrote: >>>> I've attached a test patch to the CONNECTORS-478 ticket that you can >>>> download and use to patch 0.5.1 sources. The test patch *should* >>>> fix the problem, but if it doesn't, it will certainly log more >>>> diagnostics, enough so I can see what the change in behavior has >>>> been for Derby in this release. >>>> >>>> FWIW, it looks like Derby is throwing a slightly different kind of >>>> exception for this case than it did before. >>>> >>>> Can you please try it out and let me know what happens? >>>> >>>> Thanks! >>>> Karl >>>> >>>> >>>> On Mon, Jun 4, 2012 at 6:29 AM, Karl Wright <[email protected]> wrote: >>>>> Thanks, this is exactly what I was looking for. >>>>> >>>>> I'll put it in the ticket and have a look at this later today. >>>>> >>>>> Karl >>>>> >>>>> >>>>> On Mon, Jun 4, 2012 at 6:20 AM, Marcin Goss <[email protected]> >>>>> wrote: >>>>>> This is all I've got. I am sorry for the polish fragments, I can't seem >>>>>> to change it anywhere. Anyway, hopefully you should be able to get all >>>>>> the information you need. >>>>>> >>>>>> ERROR 2012-06-04 11:22:25,449 (Worker thread '42') - Worker thread >>>>>> aborting and restarting due to database connection reset: Database >>>>>> exception: Exception doing query: Instrukcja została przerwana, ponieważ >>>>>> mogła spowodować powstanie duplikatu wartości klucza w regule >>>>>> ograniczającej klucz unikalny lub podstawowy albo w unikalnym indeksie >>>>>> identyfikowanym przez obiekt 'C1338469165883' zdefiniowany dla 'EVENTS'. >>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Database >>>>>> exception: Exception doing query: Instrukcja została przerwana, ponieważ >>>>>> mogła spowodować powstanie duplikatu wartości klucza w regule >>>>>> ograniczającej klucz unikalny lub podstawowy albo w unikalnym indeksie >>>>>> identyfikowanym przez obiekt 'C1338469165883' zdefiniowany dla 'EVENTS'. >>>>>> at >>>>>> org.apache.manifoldcf.core.database.Database.executeViaThread(Data >>>>>> ba >>>>>> se.java:680) >>>>>> at >>>>>> org.apache.manifoldcf.core.database.Database.executeUncachedQuery( >>>>>> Da >>>>>> tabase.java:726) >>>>>> at >>>>>> org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.cr >>>>>> ea >>>>>> te(Database.java:1393) >>>>>> at >>>>>> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAn >>>>>> dE >>>>>> xecute(CacheManager.java:144) >>>>>> at >>>>>> org.apache.manifoldcf.core.database.Database.executeQuery(Database >>>>>> .j >>>>>> ava:185) >>>>>> at >>>>>> org.apache.manifoldcf.core.database.DBInterfaceDerby.performModifi >>>>>> ca >>>>>> tion(DBInterfaceDerby.java:831) >>>>>> at >>>>>> org.apache.manifoldcf.core.database.DBInterfaceDerby.performInsert >>>>>> (D >>>>>> BInterfaceDerby.java:222) >>>>>> at >>>>>> org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTa >>>>>> bl >>>>>> e.java:68) >>>>>> at >>>>>> org.apache.manifoldcf.crawler.jobs.EventManager.createEvent(EventM >>>>>> an >>>>>> ager.java:123) >>>>>> at >>>>>> org.apache.manifoldcf.crawler.jobs.JobManager.beginEventSequence(J >>>>>> ob >>>>>> Manager.java:4217) >>>>>> at >>>>>> org.apache.manifoldcf.crawler.system.WorkerThread$VersionActivity. >>>>>> be >>>>>> ginEventSequence(WorkerThread.java:1236) >>>>>> at >>>>>> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConn >>>>>> ec >>>>>> tor.lookupIPAddress(WebcrawlerConnector.java:4737) >>>>>> at >>>>>> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConn >>>>>> ec >>>>>> tor.getDocumentVersions(WebcrawlerConnector.java:634) >>>>>> at >>>>>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread >>>>>> .j >>>>>> ava:318) Caused by: >>>>>> java.sql.SQLIntegrityConstraintViolationException: Instrukcja została >>>>>> przerwana, ponieważ mogła spowodować powstanie duplikatu wartości klucza >>>>>> w regule ograniczającej klucz unikalny lub podstawowy albo w unikalnym >>>>>> indeksie identyfikowanym przez obiekt 'C1338469165883' zdefiniowany dla >>>>>> 'EVENTS'. >>>>>> at >>>>>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(U >>>>>> nk >>>>>> nown Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLExcept >>>>>> io >>>>>> n(Unknown Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException >>>>>> (U >>>>>> nknown Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement >>>>>> (U >>>>>> nknown Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Un >>>>>> kn >>>>>> own Source) >>>>>> at >>>>>> org.apache.manifoldcf.core.database.Database.execute(Database.java >>>>>> :8 >>>>>> 39) >>>>>> at >>>>>> org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.ru >>>>>> n( >>>>>> Database.java:640) Caused by: java.sql.SQLException: Instrukcja >>>>>> została przerwana, ponieważ mogła spowodować powstanie duplikatu >>>>>> wartości klucza w regule ograniczającej klucz unikalny lub podstawowy >>>>>> albo w unikalnym indeksie identyfikowanym przez obiekt 'C1338469165883' >>>>>> zdefiniowany dla 'EVENTS'. >>>>>> at >>>>>> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unk >>>>>> no >>>>>> wn Source) >>>>>> at >>>>>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransp >>>>>> or >>>>>> tAcrossDRDA(Unknown Source) >>>>>> ... 11 more >>>>>> Caused by: ERROR 23505: Instrukcja została przerwana, ponieważ mogła >>>>>> spowodować powstanie duplikatu wartości klucza w regule ograniczającej >>>>>> klucz unikalny lub podstawowy albo w unikalnym indeksie identyfikowanym >>>>>> przez obiekt 'C1338469165883' zdefiniowany dla 'EVENTS'. >>>>>> at >>>>>> org.apache.derby.iapi.error.StandardException.newException(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.execute.IndexChanger.insertAndCheckDups( >>>>>> Un >>>>>> known Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.execute.IndexChanger.doInsert(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.execute.IndexChanger.insert(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.execute.IndexSetChanger.insert(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.execute.RowChangerImpl.insertRow(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.execute.InsertResultSet.normalInsertCore >>>>>> (U >>>>>> nknown Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown >>>>>> Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unk >>>>>> no >>>>>> wn Source) >>>>>> at >>>>>> org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown >>>>>> Source) >>>>>> ... 5 more >>>>>> >>>>>> >>>>>> /Marcin >>>>>> >>>>>> -----Original Message----- >>>>>> From: Karl Wright [mailto:[email protected]] >>>>>> Sent: Monday, June 04, 2012 12:17 PM >>>>>> To: [email protected] >>>>>> Subject: Re: Derby DB up-to-date? >>>>>> >>>>>> I've created CONNECTORS-478 to track this issue. But please do send >>>>>> along the rest of the stack trace context because without that I cannot >>>>>> know where, exactly, the constraint violation is not getting caught. >>>>>> >>>>>> Thanks, >>>>>> Karl >>>>>> >>>>>> On Mon, Jun 4, 2012 at 6:08 AM, Karl Wright <[email protected]> wrote: >>>>>>> Can you provide the rest of the exception? (The "caused by" >>>>>>> clauses >>>>>>> etc.?) >>>>>>> >>>>>>> The fact is that we discovered that 10.8.2.2 throws deadlock and >>>>>>> constraint violation exceptions in different places (as compared >>>>>>> to >>>>>>> 10.7.1.1) under multi-threaded conditions. That was part of the >>>>>>> reason why the 0.5.1 release was necessary. It's of course >>>>>>> possible that we missed some places that (for instance) the wiki >>>>>>> connector especially tweaks. >>>>>>> >>>>>>> Karl >>>>>>> >>>>>>> >>>>>>> On Mon, Jun 4, 2012 at 5:29 AM, Marcin Goss <[email protected]> >>>>>>> wrote: >>>>>>>> Hello Karl, >>>>>>>> >>>>>>>> Thank you for answering. >>>>>>>> I am using Manifold 0.5.1 and what I am doing is a simple indexing of >>>>>>>> a wiki page. The error that I am getting looks like this: >>>>>>>> >>>>>>>> Caused by: java.sql.SQLIntegrityConstraintViolationException: The >>>>>>>> statement was aborted because it would have caused a duplicate key >>>>>>>> value in a unique or primary key constraint or unique index identified >>>>>>>> by 'XXXXXXXX' defined on 'EVENTS'. >>>>>>>> at >>>>>>>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException >>>>>>>> (U >>>>>>>> nkn >>>>>>>> own Source) >>>>>>>> at >>>>>>>> org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown >>>>>>>> Source) >>>>>>>> at >>>>>>>> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLExce >>>>>>>> pt >>>>>>>> ion >>>>>>>> (Unknown Source) >>>>>>>> at >>>>>>>> org.apache.derby.impl.jdbc.TransactionResourceImpl.handleExcepti >>>>>>>> on >>>>>>>> (Un >>>>>>>> known Source) >>>>>>>> at >>>>>>>> org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unkno >>>>>>>> wn >>>>>>>> Source) >>>>>>>> at >>>>>>>> org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unkno >>>>>>>> wn >>>>>>>> Source) >>>>>>>> at >>>>>>>> org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unkno >>>>>>>> wn >>>>>>>> Source) >>>>>>>> >>>>>>>> /Marcin >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: Karl Wright [mailto:[email protected]] >>>>>>>> Sent: Wednesday, May 30, 2012 12:42 PM >>>>>>>> To: [email protected] >>>>>>>> Subject: Re: Derby DB up-to-date? >>>>>>>> >>>>>>>> Also, the ManifoldCF tests passed for these releases, so I'm wondering >>>>>>>> exactly what you are seeing. Can you provide more details, including >>>>>>>> exceptions? >>>>>>>> >>>>>>>> Karl >>>>>>>> >>>>>>>> On Wed, May 30, 2012 at 6:38 AM, Karl Wright <[email protected]> >>>>>>>> wrote: >>>>>>>>> Which version of ManifoldCF are you using? The 0.5 release uses >>>>>>>>> 10.8.2.2. >>>>>>>>> >>>>>>>>> Karl >>>>>>>>> >>>>>>>>> On Wed, May 30, 2012 at 5:21 AM, Marcin Goss >>>>>>>>> <[email protected]> wrote: >>>>>>>>>> I'm getting an error in a default manifoldCF setup (jetty >>>>>>>>>> server and derby >>>>>>>>>> db) that is probably caused (my supposition) by this bug in Derby: >>>>>>>>>> https://issues.apache.org/jira/browse/DERBY-3343. The bug was >>>>>>>>>> fixed in January 2011. Which version of Derby db is coming with >>>>>>>>>> ManifoldCF? >>>>>>>>>> Is it the one with this bug fixed? >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>> >>> > >
