[
https://issues.apache.org/jira/browse/OAK-10758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17924639#comment-17924639
]
José Andrés Cordero Benítez commented on OAK-10758:
---------------------------------------------------
Closing this ticket, as the issue seems not reproducible and the FullGC code
has changed so much that this is not relevant anymore.
> Avoid or mitigate Cursor Timeout in Mongo when running detailedGC
> -----------------------------------------------------------------
>
> Key: OAK-10758
> URL: https://issues.apache.org/jira/browse/OAK-10758
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk, oak-run
> Reporter: José Andrés Cordero Benítez
> Assignee: José Andrés Cordero Benítez
> Priority: Major
>
> When running oak-run revisions detailedGC command, if the cleanup is too big
> the processing time can exceed the Mongo cursor timeout, which defaults to 30
> minutes:
> {code:java}
> java.io.IOException: com.mongodb.MongoCursorNotFoundException: Query failed
> with error code -5 and error message 'Cursor 1007212049743349339 not found on
> server mongohost' on server mongohost:27017
> at
> org.apache.jackrabbit.oak.run.RevisionsCommand.collectOnce(RevisionsCommand.java:427)
> at
> org.apache.jackrabbit.oak.run.RevisionsCommand.collect(RevisionsCommand.java:377)
> at
> org.apache.jackrabbit.oak.run.RevisionsCommand.execute(RevisionsCommand.java:245)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> Caused by: com.mongodb.MongoCursorNotFoundException: Query failed with error
> code -5 and error message 'Cursor 1007212049743349339 not found on server
> mongohost:27017' on server mongohost:27017
> at
> com.mongodb.operation.QueryHelper.translateCommandException(QueryHelper.java:27)
> at
> com.mongodb.operation.QueryBatchCursor.getMore(QueryBatchCursor.java:268)
> at
> com.mongodb.operation.QueryBatchCursor.hasNext(QueryBatchCursor.java:139)
> at
> com.mongodb.client.internal.MongoBatchCursorAdapter.hasNext(MongoBatchCursorAdapter.java:54)
> at
> org.apache.jackrabbit.guava.common.collect.TransformedIterator.hasNext(TransformedIterator.java:46)
> at
> org.apache.jackrabbit.guava.common.collect.Iterators$5.computeNext(Iterators.java:670)
> at
> org.apache.jackrabbit.guava.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:145)
> at
> org.apache.jackrabbit.guava.common.collect.AbstractIterator.hasNext(AbstractIterator.java:140)
> at
> org.apache.jackrabbit.guava.common.collect.Iterators$ConcatenatedIterator.hasNext(Iterators.java:1395)
> at
> org.apache.jackrabbit.oak.plugins.document.SplitDocumentCleanUp.disconnect(SplitDocumentCleanUp.java:63)
> at
> org.apache.jackrabbit.oak.plugins.document.VersionGCSupport.deleteSplitDocuments(VersionGCSupport.java:146)
> at
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector$GCJob.collectSplitDocuments(VersionGarbageCollector.java:842)
> at
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector$GCJob.gc(VersionGarbageCollector.java:686)
> at
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector$GCJob.run(VersionGarbageCollector.java:653)
> at
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector.gc(VersionGarbageCollector.java:221)
> at
> org.apache.jackrabbit.oak.run.RevisionsCommand$2.call(RevisionsCommand.java:401)
> at
> org.apache.jackrabbit.oak.run.RevisionsCommand$2.call(RevisionsCommand.java:398)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> 15:45:08.184 [DocumentNodeStore cluster update thread (0)] WARN
> o.a.j.o.p.document.DocumentNodeStore - Background operation failed:
> java.lang.reflect.UndeclaredThrowableException
> java.lang.reflect.UndeclaredThrowableException: null
> at com.sun.proxy.$Proxy0.query(Unknown Source)
> at
> org.apache.jackrabbit.oak.plugins.document.ClusterNodeInfoDocument.all(ClusterNodeInfoDocument.java:154)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.updateClusterState(DocumentNodeStore.java:2480)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore$BackgroundClusterUpdate.execute(DocumentNodeStore.java:3931)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore$NodeStoreTask.run(DocumentNodeStore.java:3734)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.lang.reflect.InvocationTargetException: null
> at jdk.internal.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.apache.jackrabbit.oak.plugins.document.util.ReadOnlyDocumentStoreWrapperFactory$1.invoke(ReadOnlyDocumentStoreWrapperFactory.java:41)
> ... 6 common frames omitted
> Caused by: java.lang.IllegalStateException: The pool is closed
> at
> com.mongodb.internal.connection.ConcurrentPool.get(ConcurrentPool.java:137)
> at
> com.mongodb.internal.connection.ConcurrentPool.get(ConcurrentPool.java:123)
> at
> com.mongodb.internal.session.ServerSessionPool.get(ServerSessionPool.java:80)
> at
> com.mongodb.internal.session.BaseClientSessionImpl.<init>(BaseClientSessionImpl.java:44)
> at
> com.mongodb.client.internal.ClientSessionImpl.<init>(ClientSessionImpl.java:56)
> at
> com.mongodb.client.internal.MongoClientDelegate.createClientSession(MongoClientDelegate.java:120)
> at com.mongodb.Mongo.createClientSession(Mongo.java:837)
> at com.mongodb.MongoClient.startSession(MongoClient.java:582)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoSessionFactory.createClientSession(MongoSessionFactory.java:55)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDBConnection.createClientSession(MongoDBConnection.java:115)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createClientSession(MongoDocumentStore.java:2405)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.execute(MongoDocumentStore.java:2389)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.queryInternal(MongoDocumentStore.java:846)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.queryWithRetry(MongoDocumentStore.java:774)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.query(MongoDocumentStore.java:751)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.query(MongoDocumentStore.java:739)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.query(MongoDocumentStore.java:728)
> ... 10 common frames omitted {code}
> Check if this needs mitigation or could happen as part of the internal
> repository detailedGC execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
