[jira] [Commented] (OAK-10654) Build Jackrabbit/jackrabbit-oak-trunk #1363 failed
[ https://issues.apache.org/jira/browse/OAK-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819343#comment-17819343 ] Hudson commented on OAK-10654: -- Previously failing build now is OK. Passed run: [Jackrabbit/jackrabbit-oak-trunk #1372|https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/1372/] [console log|https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/1372/console] > Build Jackrabbit/jackrabbit-oak-trunk #1363 failed > -- > > Key: OAK-10654 > URL: https://issues.apache.org/jira/browse/OAK-10654 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson >Priority: Major > > No description is provided > The build Jackrabbit/jackrabbit-oak-trunk #1363 has failed. > First failed run: [Jackrabbit/jackrabbit-oak-trunk > #1363|https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/1363/] > [console > log|https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/1363/console] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10657) MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB limit
[ https://issues.apache.org/jira/browse/OAK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819295#comment-17819295 ] Manfred Baedke commented on OAK-10657: -- 6. is actually quite cool. > MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB > limit > --- > > Key: OAK-10657 > URL: https://issues.apache.org/jira/browse/OAK-10657 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > > To address the 16MB/childorder issue, there are many potential approaches: > - make GC more aggressive > - try to change updates to remove "in-between" changes of ":childOrder" > property > - change the data model of ":childOrder" > - try to shrink document in DB once size related exception happens > This ticket is about the last of these options. > Proposal: > - improve exception thrown by document store so that it can be acted upon > - in document store utils add a method that inspects a document and produces > UpdateOps suitable to shrink the document > - DocumentNodeStore commit could catch exception, obtain update ops, apply > them, and retry once (this should be dependant on a feature toggle) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OAK-10657) MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB limit
[ https://issues.apache.org/jira/browse/OAK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819280#comment-17819280 ] Stefan Egli edited comment on OAK-10657 at 2/21/24 2:46 PM: Or to put a bit simpler: * 5. would be after-commit large property checks - in the background * 6. would be heuristic based before-commit large property checks while * 4. would be on-exception large property check - on the spot was (Author: egli): Or to put a bit simpler: * 5. would be after-commit large property checks - in the background * 6. would be heuristic based before-commit large property checks > MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB > limit > --- > > Key: OAK-10657 > URL: https://issues.apache.org/jira/browse/OAK-10657 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > > To address the 16MB/childorder issue, there are many potential approaches: > - make GC more aggressive > - try to change updates to remove "in-between" changes of ":childOrder" > property > - change the data model of ":childOrder" > - try to shrink document in DB once size related exception happens > This ticket is about the last of these options. > Proposal: > - improve exception thrown by document store so that it can be acted upon > - in document store utils add a method that inspects a document and produces > UpdateOps suitable to shrink the document > - DocumentNodeStore commit could catch exception, obtain update ops, apply > them, and retry once (this should be dependant on a feature toggle) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10657) MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB limit
[ https://issues.apache.org/jira/browse/OAK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819280#comment-17819280 ] Stefan Egli commented on OAK-10657: --- Or to put a bit simpler: * 5. would be after-commit large property checks - in the background * 6. would be heuristic based before-commit large property checks > MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB > limit > --- > > Key: OAK-10657 > URL: https://issues.apache.org/jira/browse/OAK-10657 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > > To address the 16MB/childorder issue, there are many potential approaches: > - make GC more aggressive > - try to change updates to remove "in-between" changes of ":childOrder" > property > - change the data model of ":childOrder" > - try to shrink document in DB once size related exception happens > This ticket is about the last of these options. > Proposal: > - improve exception thrown by document store so that it can be acted upon > - in document store utils add a method that inspects a document and produces > UpdateOps suitable to shrink the document > - DocumentNodeStore commit could catch exception, obtain update ops, apply > them, and retry once (this should be dependant on a feature toggle) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10657) MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB limit
[ https://issues.apache.org/jira/browse/OAK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819279#comment-17819279 ] Stefan Egli commented on OAK-10657: --- So for reading simplicity I guess this in the context of the stack trace mentioned in [another comment|https://issues.apache.org/jira/browse/OAK-10642?focusedCommentId=17819206=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17819206] ? Then I'd say having a new type would be feasible. That (the handler code of that new exception type) could then invoke the "util" that can provide UpdateOps for cleanup (which itself could be in or near "DetailedGC territory" - as that could be needed there too). And I'm guessing we're looking at doing it at exception time because ... we don't have the (before) document at hand for inspecting large properties _before_ we're sending the update? So having an "exception time" handling (this 4th approach) sounds feasible. Having said that, I'm wondering about some other approaches: * 5. upon successful branch commit we'd have the actual before document(s), so we could inspect them for large properties. We could treat that similar to how splits are handled (Commit.checkSplitCandidate). That might be what is already covered under the 1st appraoch though (make GC more aggressive) ... * 6. in a variation to 5.: could we start collecting large (eg 1k) property updates in the branch (eg DocumentNodeStoreBranch) after a successful branch commit - so that in case we'll update that same property in the same branch again, we'd actually have some infos and could proactively remove the previous branch commit, besides having applying the new one. This could catch a majority of cases in a non intrusive way. > MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB > limit > --- > > Key: OAK-10657 > URL: https://issues.apache.org/jira/browse/OAK-10657 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > > To address the 16MB/childorder issue, there are many potential approaches: > - make GC more aggressive > - try to change updates to remove "in-between" changes of ":childOrder" > property > - change the data model of ":childOrder" > - try to shrink document in DB once size related exception happens > This ticket is about the last of these options. > Proposal: > - improve exception thrown by document store so that it can be acted upon > - in document store utils add a method that inspects a document and produces > UpdateOps suitable to shrink the document > - DocumentNodeStore commit could catch exception, obtain update ops, apply > them, and retry once (this should be dependant on a feature toggle) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10657) MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB limit
[ https://issues.apache.org/jira/browse/OAK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819236#comment-17819236 ] Julian Reschke commented on OAK-10657: -- Looking at https://github.com/apache/jackrabbit-oak/blob/trunk/oak-store-document/src/main/java/org/apache/jackrabbit/oak/plugins/document/DocumentStoreException.java ... We apparently decided not to use a hierarchy of exceptions, but just to use message + type. Type can be GENERIC or TRANSIENT to allow retries. That doesn't help here, as the operation only becomes retryable after cleaning up the document. Choices: - add a new type (FWIW, an API change) - inspect mesage or cause (in the latter case we'd need to invent something for testing with MemoryDocumentStore) - leave the handling to the actual store implementation Feedback appreciated... > MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB > limit > --- > > Key: OAK-10657 > URL: https://issues.apache.org/jira/browse/OAK-10657 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > > To address the 16MB/childorder issue, there are many potential approaches: > - make GC more aggressive > - try to change updates to remove "in-between" changes of ":childOrder" > property > - change the data model of ":childOrder" > - try to shrink document in DB once size related exception happens > This ticket is about the last of these options. > Proposal: > - improve exception thrown by document store so that it can be acted upon > - in document store utils add a method that inspects a document and produces > UpdateOps suitable to shrink the document > - DocumentNodeStore commit could catch exception, obtain update ops, apply > them, and retry once (this should be dependant on a feature toggle) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10656) MongoDocumentStore: keep metrics about document size related exceptions
[ https://issues.apache.org/jira/browse/OAK-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819214#comment-17819214 ] Julian Reschke commented on OAK-10656: -- trunk: [2e996d78f0|https://github.com/apache/jackrabbit-oak/commit/2e996d78f0a565b17287af5691f2c1be7d2e925d] > MongoDocumentStore: keep metrics about document size related exceptions > --- > > Key: OAK-10656 > URL: https://issues.apache.org/jira/browse/OAK-10656 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > Fix For: 1.62.0 > > > It might be useful to have metrics about the amount of BSON/document size > related excepttions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10656) MongoDocumentStore: keep metrics about document size related exceptions
[ https://issues.apache.org/jira/browse/OAK-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-10656. -- Fix Version/s: 1.62.0 Resolution: Fixed > MongoDocumentStore: keep metrics about document size related exceptions > --- > > Key: OAK-10656 > URL: https://issues.apache.org/jira/browse/OAK-10656 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > Fix For: 1.62.0 > > > It might be useful to have metrics about the amount of BSON/document size > related excepttions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OAK-10657) MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB limit
[ https://issues.apache.org/jira/browse/OAK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke reassigned OAK-10657: Assignee: Julian Reschke > MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB > limit > --- > > Key: OAK-10657 > URL: https://issues.apache.org/jira/browse/OAK-10657 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > > To address the 16MB/childorder issue, there are many potential approaches: > - make GC more aggressive > - try to change updates to remove "in-between" changes of ":childOrder" > property > - change the data model of ":childOrder" > - try to shrink document in DB once size related exception happens > This ticket is about the last of these options. > Proposal: > - improve exception thrown by document store so that it can be acted upon > - in document store utils add a method that inspects a document and produces > UpdateOps suitable to shrink the document > - DocumentNodeStore commit could catch exception, obtain update ops, apply > them, and retry once (this should be dependant on a feature toggle) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10642) Add tests for operations on very large ordered collections
[
https://issues.apache.org/jira/browse/OAK-10642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819206#comment-17819206
]
Julian Reschke commented on OAK-10642:
--
FTR:
{noformat}
mvn clean install -Dtest=ManyChildrenIT -Dupdate.limit=100
-Dnsfixtures="DOCUMENT_NS"
{noformat}
(with tests re-enabled) fails for me with:
{noformat}
[INFO] Running org.apache.jackrabbit.oak.jcr.ManyChildrenIT
[ERROR] Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 179.658
s <<< FAILURE! - in org.apache.jackrabbit.oak.jcr.ManyChildrenIT
[ERROR] copyOrderableWithManyChildren[DocumentNodeStore[Mongo] on
mongodb://localhost:56800/MongoMKDB?connectTimeoutMS=3000=3000](org.apache.jackrabbit.oak.jcr.ManyChildrenIT)
Time elapsed: 27.483 s <<< ERROR!
org.apache.jackrabbit.oak.plugins.document.DocumentStoreException: Write
operation error on server localhost:56800. Write error: WriteError{code=10334,
message='BSONObj size: 16981429 (0x1031DB5) is invalid. Size must be between 0
and 16793600(16MB) First element: _id: "1:/test-1"', details={}}. [1:/test-1]
at
org.apache.jackrabbit.oak.plugins.document.DocumentStoreException.asDocumentStoreException(DocumentStoreException.java:179)
at
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.handleException(MongoDocumentStore.java:2328)
at
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.handleException(MongoDocumentStore.java:2339)
at
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.findAndModify(MongoDocumentStore.java:1156)
at
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createOrUpdate(MongoDocumentStore.java:1282)
at
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createOrUpdate(MongoDocumentStore.java:1361)
at
org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDocumentStoreWrapper.createOrUpdate(LeaseCheckDocumentStoreWrapper.java:144)
at
org.apache.jackrabbit.oak.plugins.document.Commit.applyToDocumentStore(Commit.java:371)
at
org.apache.jackrabbit.oak.plugins.document.Commit.applyToDocumentStoreWithTiming(Commit.java:278)
at
org.apache.jackrabbit.oak.plugins.document.Commit.prepare(Commit.java:245)
at
org.apache.jackrabbit.oak.plugins.document.Commit.apply(Commit.java:209)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.persist(DocumentNodeStoreBranch.java:321)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.persist(DocumentNodeStoreBranch.java:283)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch$Persisted.persistTransientHead(DocumentNodeStoreBranch.java:719)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch$Persisted.setRoot(DocumentNodeStoreBranch.java:665)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.setRoot(DocumentNodeStoreBranch.java:113)
at
org.apache.jackrabbit.oak.plugins.document.DocumentRootBuilder.purge(DocumentRootBuilder.java:185)
at
org.apache.jackrabbit.oak.plugins.document.DocumentRootBuilder.updated(DocumentRootBuilder.java:105)
at
org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.updated(MemoryNodeBuilder.java:213)
at
org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:510)
at
org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setProperty(MemoryNodeBuilder.java:524)
at
org.apache.jackrabbit.oak.core.SecureNodeBuilder.setProperty(SecureNodeBuilder.java:271)
at
org.apache.jackrabbit.oak.plugins.tree.impl.AbstractMutableTree.setProperty(AbstractMutableTree.java:187)
at
org.apache.jackrabbit.oak.core.MutableTree.setProperty(MutableTree.java:238)
at
org.apache.jackrabbit.oak.plugins.tree.TreeUtil.addChild(TreeUtil.java:249)
at
org.apache.jackrabbit.oak.jcr.delegate.WorkspaceDelegate$WorkspaceCopy.copy(WorkspaceDelegate.java:139)
at
org.apache.jackrabbit.oak.jcr.delegate.WorkspaceDelegate$WorkspaceCopy.copy(WorkspaceDelegate.java:160)
at
org.apache.jackrabbit.oak.jcr.delegate.WorkspaceDelegate$WorkspaceCopy.perform(WorkspaceDelegate.java:121)
at
org.apache.jackrabbit.oak.jcr.delegate.WorkspaceDelegate.copy(WorkspaceDelegate.java:99)
at
org.apache.jackrabbit.oak.jcr.session.WorkspaceImpl$2.performVoid(WorkspaceImpl.java:163)
at
org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:299)
at
org.apache.jackrabbit.oak.jcr.session.WorkspaceImpl.copy(WorkspaceImpl.java:150)
at
org.apache.jackrabbit.oak.jcr.session.WorkspaceImpl.copy(WorkspaceImpl.java:132)
at
org.apache.jackrabbit.oak.jcr.ManyChildrenIT.copyOrderableWithManyChildren(ManyChildrenIT.java:142)
at
[jira] [Updated] (OAK-10657) MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB limit
[ https://issues.apache.org/jira/browse/OAK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-10657: - Description: To address the 16MB/childorder issue, there are many potential approaches: - make GC more aggressive - try to change updates to remove "in-between" changes of ":childOrder" property - change the data model of ":childOrder" - try to shrink document in DB once size related exception happens This ticket is about the last of these options. Proposal: - improve exception thrown by document store so that it can be acted upon - in document store utils. add a method that inspects a document and produces UpdateOps suitable to shrink the document - DocumentNodeStore commit could catch exception, obtain update ops, apply them, and retry once (this should be dependant on a feature toggle) was: To address the 16MB/childorder issue, there are many potential approaches: - make GC more aggressive - try to change updates to remove "in-between" changes of ":childOrder" property - change the data model of ":childOrder" - try to shrink document in DB once size related exception happens This issues is about the last of these options. Proposal: - improve exception thrown by document store so that it can be acted upon - in document store utils. add a method that inspects a document and produces UpdateOps suitable to shrink the document - DocumentNodeStore commit could catch exception, obtain update ops, apply them, and retry once (this should be dependant on a feature toggle) > MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB > limit > --- > > Key: OAK-10657 > URL: https://issues.apache.org/jira/browse/OAK-10657 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk >Reporter: Julian Reschke >Priority: Major > > To address the 16MB/childorder issue, there are many potential approaches: > - make GC more aggressive > - try to change updates to remove "in-between" changes of ":childOrder" > property > - change the data model of ":childOrder" > - try to shrink document in DB once size related exception happens > This ticket is about the last of these options. > Proposal: > - improve exception thrown by document store so that it can be acted upon > - in document store utils. add a method that inspects a document and produces > UpdateOps suitable to shrink the document > - DocumentNodeStore commit could catch exception, obtain update ops, apply > them, and retry once (this should be dependant on a feature toggle) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10657) MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB limit
[ https://issues.apache.org/jira/browse/OAK-10657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-10657: - Description: To address the 16MB/childorder issue, there are many potential approaches: - make GC more aggressive - try to change updates to remove "in-between" changes of ":childOrder" property - change the data model of ":childOrder" - try to shrink document in DB once size related exception happens This ticket is about the last of these options. Proposal: - improve exception thrown by document store so that it can be acted upon - in document store utils add a method that inspects a document and produces UpdateOps suitable to shrink the document - DocumentNodeStore commit could catch exception, obtain update ops, apply them, and retry once (this should be dependant on a feature toggle) was: To address the 16MB/childorder issue, there are many potential approaches: - make GC more aggressive - try to change updates to remove "in-between" changes of ":childOrder" property - change the data model of ":childOrder" - try to shrink document in DB once size related exception happens This ticket is about the last of these options. Proposal: - improve exception thrown by document store so that it can be acted upon - in document store utils. add a method that inspects a document and produces UpdateOps suitable to shrink the document - DocumentNodeStore commit could catch exception, obtain update ops, apply them, and retry once (this should be dependant on a feature toggle) > MongoDocumentStore: shrink in-DB documents after updates fail due to 16MB > limit > --- > > Key: OAK-10657 > URL: https://issues.apache.org/jira/browse/OAK-10657 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk >Reporter: Julian Reschke >Priority: Major > > To address the 16MB/childorder issue, there are many potential approaches: > - make GC more aggressive > - try to change updates to remove "in-between" changes of ":childOrder" > property > - change the data model of ":childOrder" > - try to shrink document in DB once size related exception happens > This ticket is about the last of these options. > Proposal: > - improve exception thrown by document store so that it can be acted upon > - in document store utils add a method that inspects a document and produces > UpdateOps suitable to shrink the document > - DocumentNodeStore commit could catch exception, obtain update ops, apply > them, and retry once (this should be dependant on a feature toggle) -- This message was sent by Atlassian Jira (v8.20.10#820010)
