[jira] [Commented] (OAK-2609) Thread.interrupt seems to stop repository
[ https://issues.apache.org/jira/browse/OAK-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363255#comment-14363255 ] Michael Dürig commented on OAK-2609: I don't think we should do anything here on the Oak side as we are dealing with [documented side effects|http://docs.oracle.com/javase/7/docs/api/java/nio/channels/InterruptibleChannel.html] of {{Thread.interrupt}}: Thread.interrupt seems to stop repository - Key: OAK-2609 URL: https://issues.apache.org/jira/browse/OAK-2609 Project: Jackrabbit Oak Issue Type: Bug Affects Versions: 1.1.7 Environment: Java 7 Reporter: Carsten Ziegeler Priority: Critical Fix For: 1.1.8, 1.0.13 We have a sporadic problem with Sling's JCR installer 3.3.8 and Oak (tar mk). It seems to timing related: the JCR installer does a Thread#interrupt at one point and sometimes this brings the hole instance to stop. Nothing else is going on any more. While of course, a workaround is to remove the Thread.interrupt call in the JCR installer (which we did, see SLING-4477), I have the fear that this can happen with any code that is using the repository and gets interrupted. This error is hard to reproduce, however with three people testing we could see this several times happening -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2631) Use buffered variants for IndexInput and IndexOutput
[ https://issues.apache.org/jira/browse/OAK-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-2631: - Fix Version/s: (was: 1.2) 1.1.8 Use buffered variants for IndexInput and IndexOutput Key: OAK-2631 URL: https://issues.apache.org/jira/browse/OAK-2631 Project: Jackrabbit Oak Issue Type: Improvement Components: oak-lucene Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Fix For: 1.1.8 Attachments: OAK-2631.patch Lucene provides a buffered variants for {{IndexInput}} and {{IndexOutput}}. Currently Oak extends these classes directly. For better performance itshould extend the buffered variants. As discussed [here|https://issues.apache.org/jira/browse/OAK-?focusedCommentId=14178265#comment-14178265] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2624) Move out non essential task processing from DoucmentNodeStore background thread
[ https://issues.apache.org/jira/browse/OAK-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2624: -- Fix Version/s: (was: 1.1.8) 1.1.9 Move out non essential task processing from DoucmentNodeStore background thread --- Key: OAK-2624 URL: https://issues.apache.org/jira/browse/OAK-2624 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Chetan Mehrotra Assignee: Marcel Reutegger Fix For: 1.1.9 The {{DocumentNodeStore background thread}} handles quite a few task currently like # Clean orphaned branches # Clean collisions # Split documents # Background write - write back pending updates to _lastRev # Detect change in head revision from backend # Cache invalidation # Revision Comparator purge The background operation is scheduled to be performed every second (1s frequency). However at times some of the above task take more time which delay in picking up the changes from backend. For quick propagation of changes in cluster it is essential that work done in background thread is performed quickly. To enable that we should move out task which are not directly related to _head revision change_ like splitDocument. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2478) Move spellcheck config to own configuration node
[ https://issues.apache.org/jira/browse/OAK-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2478: - Fix Version/s: (was: 1.1.8) Move spellcheck config to own configuration node Key: OAK-2478 URL: https://issues.apache.org/jira/browse/OAK-2478 Project: Jackrabbit Oak Issue Type: Improvement Components: oak-lucene Reporter: Tommaso Teofili Fix For: 1.2 Currently spellcheck configuration is controlled via properties defined on main config / props node but it'd be good if we would have its own place to configure the whole spellcheck feature to not mix up configuration of other features / parameters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1752) Node name queries should use an index
[ https://issues.apache.org/jira/browse/OAK-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-1752: - Attachment: OAK-1752.patch [patch|^OAK-1752.patch] for the same. It depends upon the patch in OAK-2634. * Node names would be index against a Lucene field named {{:nodeName}} * Supports equals and like queries * To enable this support one need to set {{indexNodeName}} to true in the indexRule for that nodeType Later we can look into possibility of using custom tokenizer to index the node name differently. For e.g. for nt:file we can configure a camelCase tokenizer to extract multiple fields. This would avoid using expensive like query to lookup file by partial names [~alexparvulescu] [~tmueller] Can you have a look and provide feedback Node name queries should use an index - Key: OAK-1752 URL: https://issues.apache.org/jira/browse/OAK-1752 Project: Jackrabbit Oak Issue Type: Improvement Components: query Reporter: Alex Parvulescu Assignee: Chetan Mehrotra Fix For: 1.2 Attachments: OAK-1752.patch The node name queries don't use any index currently, making them really slow and triggering a lot of traversal warnings. Simply adding node names to a property index would be too much content indexed, but as Lucene already indexes the node names, using this index would be one viable option. {code} /jcr:root//*[fn:name() = 'jcr:content'] /jcr:root//*[jcr:like(fn:name(), 'jcr:con%')] {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2633) Log document as debug message on conflict
[ https://issues.apache.org/jira/browse/OAK-2633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-2633. --- Resolution: Fixed Done in trunk: http://svn.apache.org/r1666942 Enable debug logging for {{org.apache.jackrabbit.oak.plugins.document.Commit}} to see details of a conflict. Log document as debug message on conflict - Key: OAK-2633 URL: https://issues.apache.org/jira/browse/OAK-2633 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Fix For: 1.1.8 The implementation currently appends the complete document to the exception message when a conflict occurs on commit. It would be better to only log the document details on DEBUG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2499) Expose mongo and db versions for reporting purposes
[ https://issues.apache.org/jira/browse/OAK-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-2499: Fix Version/s: 1.0.13 Expose mongo and db versions for reporting purposes --- Key: OAK-2499 URL: https://issues.apache.org/jira/browse/OAK-2499 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk, rdbmk, segmentmk Reporter: Marius Petria Assignee: Chetan Mehrotra Fix For: 1.1.8, 1.0.13 Attachments: OAK-2499.1.diff, OAK-2499.2.diff For reporting purposes I need to find out from java the MK type and also for mongo version and the info about used DB make/version? [~chetanm] suggested that such information can be exposed when we register a {{NodeStore}} instances within OSGi we can also add a service property which captures such implementation details. Possibly use service.description or define a new one which provide a comma separated list of attributes.” -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2610) Annotate intermediate docs with property names
[ https://issues.apache.org/jira/browse/OAK-2610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2610: -- Fix Version/s: (was: 1.1.8) 1.3.0 Annotate intermediate docs with property names -- Key: OAK-2610 URL: https://issues.apache.org/jira/browse/OAK-2610 Project: Jackrabbit Oak Issue Type: Improvement Reporter: Marcel Reutegger Assignee: Marcel Reutegger Fix For: 1.3.0 Reading through a ValueMap can be very inefficient if the changes of a given property are distributed sparsely across the previous documents. The current implementation has to scan through the entire set of previous documents to collect the changes. Intermediate documents should have additional information about what properties are present on referenced previous documents. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2574) Update mongo-java-driver to 2.13.0
[ https://issues.apache.org/jira/browse/OAK-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-2574. --- Resolution: Fixed Done in trunk: http://svn.apache.org/r1666957 Update mongo-java-driver to 2.13.0 -- Key: OAK-2574 URL: https://issues.apache.org/jira/browse/OAK-2574 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Fix For: 1.1.8 MongoDB 3.0 was released yesterday and the current java driver (2.12.2) used by Oak is marked as untested with 3.0: http://docs.mongodb.org/ecosystem/drivers/java/#compatibility We should update to 2.13.0, which is compatible with 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2420) DocumentNodeStore revision GC may lead to NPE
[ https://issues.apache.org/jira/browse/OAK-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2420: -- Fix Version/s: (was: 1.1.8) 1.1.9 DocumentNodeStore revision GC may lead to NPE - Key: OAK-2420 URL: https://issues.apache.org/jira/browse/OAK-2420 Project: Jackrabbit Oak Issue Type: Bug Components: core Affects Versions: 1.0 Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Fix For: 1.1.9 The DocumentNodeStore revision GC may cause a NPE in a reader thread when the GC deletes documents currently accessed by the reader. The {{docChildrenCache}} is invalidated in {{VersionGarbageCollector.collectDeletedDocuments()}} after documents are removed in the DocumentStore. The NPE may occur if removed documents are access in between. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1980) Use index on non-root node
[ https://issues.apache.org/jira/browse/OAK-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella updated OAK-1980: -- Fix Version/s: 1.4 Use index on non-root node -- Key: OAK-1980 URL: https://issues.apache.org/jira/browse/OAK-1980 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Marcel Reutegger Assignee: Davide Giannella Fix For: 1.4 Attachments: OAK-1980.patch Oak is able to maintain indexes on any location in the hierarchy. However the lookup for most index implementations only make use of an index under the root node. There are various TODOs in the code regarding this, e.g. in PropertyIndex. Looking up an index along the filter path adds some additional cost, but should be within reasonable bounds. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2476) Move our CI to Jenkins
[ https://issues.apache.org/jira/browse/OAK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363079#comment-14363079 ] Tommaso Teofili commented on OAK-2476: -- right, I hope 1G is enough, added -DargLine=-Xmx1024m to the Maven command. Move our CI to Jenkins -- Key: OAK-2476 URL: https://issues.apache.org/jira/browse/OAK-2476 Project: Jackrabbit Oak Issue Type: Task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Priority: Critical Labels: CI, build, infrastructure Fix For: 1.2 We should strive for stabilization of our CI setup, as of now we had Buildbot and Travis. It seems ASF Jenkins can perform jobs on different environments (*nix, Windows and others) so we can evaluate that and check if it better address our needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (OAK-2476) Move our CI to Jenkins
[ https://issues.apache.org/jira/browse/OAK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2476: - Comment: was deleted (was: right, I hope 1G is enough, added -DargLine=-Xmx1024m to the Maven command.) Move our CI to Jenkins -- Key: OAK-2476 URL: https://issues.apache.org/jira/browse/OAK-2476 Project: Jackrabbit Oak Issue Type: Task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Priority: Critical Labels: CI, build, infrastructure Fix For: 1.2 We should strive for stabilization of our CI setup, as of now we had Buildbot and Travis. It seems ASF Jenkins can perform jobs on different environments (*nix, Windows and others) so we can evaluate that and check if it better address our needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2476) Move our CI to Jenkins
[ https://issues.apache.org/jira/browse/OAK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363080#comment-14363080 ] Tommaso Teofili commented on OAK-2476: -- right, I hope 1G is enough, added -DargLine=-Xmx1024m to the Maven command. Move our CI to Jenkins -- Key: OAK-2476 URL: https://issues.apache.org/jira/browse/OAK-2476 Project: Jackrabbit Oak Issue Type: Task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Priority: Critical Labels: CI, build, infrastructure Fix For: 1.2 We should strive for stabilization of our CI setup, as of now we had Buildbot and Travis. It seems ASF Jenkins can perform jobs on different environments (*nix, Windows and others) so we can evaluate that and check if it better address our needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2596) more (jmx) instrumentation for observation queue
[ https://issues.apache.org/jira/browse/OAK-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2596: --- Priority: Blocker (was: Major) more (jmx) instrumentation for observation queue Key: OAK-2596 URL: https://issues.apache.org/jira/browse/OAK-2596 Project: Jackrabbit Oak Issue Type: Improvement Components: core Affects Versions: 1.0.12 Reporter: Stefan Egli Assignee: Michael Dürig Priority: Blocker Labels: monitoring, observation Fix For: 1.1.8 While debugging issues with the observation queue it would be handy to have more detailed information available. At the moment you can only see one value wrt length of the queue: that is the maximum of all queues. It is unclear if the queue is that long for only one or many listeners. And it is unclear from that if the listener is slow or the engine that produces the events for the listener. So I'd suggest to add the following details - possible exposed via JMX? : # add queue length details to each of the observation listeners # have a history of the last, eg 1000 events per listener showing a) how long the event took to be created/generated and b) how long the listener took to process. Sometimes averages are not detailed enough so such a in-depth information might become useful. (Not sure about the feasibility of '1000' here - maybe that could be configurable though - just putting the idea out here). # have some information about whether a listener is currently 'reading events from the cache' or whether it has to go to eg mongo # maybe have a 'top 10' listeners that have the largest queue at the moment to easily allow navigation instead of having to go through all (eg 200) listeners manually each time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2499) Expose mongo and db versions for reporting purposes
[ https://issues.apache.org/jira/browse/OAK-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363163#comment-14363163 ] Julian Reschke commented on OAK-2499: - Will merge into 1.0. Expose mongo and db versions for reporting purposes --- Key: OAK-2499 URL: https://issues.apache.org/jira/browse/OAK-2499 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk, rdbmk, segmentmk Reporter: Marius Petria Assignee: Chetan Mehrotra Fix For: 1.1.8 Attachments: OAK-2499.1.diff, OAK-2499.2.diff For reporting purposes I need to find out from java the MK type and also for mongo version and the info about used DB make/version? [~chetanm] suggested that such information can be exposed when we register a {{NodeStore}} instances within OSGi we can also add a service property which captures such implementation details. Possibly use service.description or define a new one which provide a comma separated list of attributes.” -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2635) TimeSeriesMax's frequent 'drops to 0'
[ https://issues.apache.org/jira/browse/OAK-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363491#comment-14363491 ] Michael Dürig commented on OAK-2635: AFAICS {{TimeSeriesAverage}} has the same problem. In the end this is cause by missing values, so I think we should add a way to expose such. TimeSeriesMax's frequent 'drops to 0' - Key: OAK-2635 URL: https://issues.apache.org/jira/browse/OAK-2635 Project: Jackrabbit Oak Issue Type: Bug Components: core Affects Versions: 1.0.12 Reporter: Stefan Egli The current implementation of TimeSeriesMax - which is what is backing eg the very important 'ObservationQueueMaxLength' statistics - has a very infamous behavior: it does very frequent, intermittent 'jumps back to 0'. This even though the queue-lengths are still at the previous highs, as can often be seen with subsequent measurements (which eg are still showing there are 1000 events in the observation queue). The reason seems to be that * the value is increased via {{TimeSeriesMax.recordValue()}} during a 1 second interval * reset to 0 via {{TimeSeriesMaxinit.run()}} every second So basically, every second the counter is reset, then during 1 second if any call to {{recordValue()}} happens, it is increased. This in my view is rather unfortunate - as it can result in mentioned 'jumpy-0' behavior, but it can also jump to values in between if the largest queue does not reports its length during 1 second. It sounds a bit like this was done this way intentionally? (perhaps to make it as inexpensive as possible) or could this be fixed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2636) Issues with Maximum node name size and path size
Will McGauley created OAK-2636: -- Summary: Issues with Maximum node name size and path size Key: OAK-2636 URL: https://issues.apache.org/jira/browse/OAK-2636 Project: Jackrabbit Oak Issue Type: Bug Components: doc, mongomk Affects Versions: 1.1.7 Reporter: Will McGauley I ran across the maximum allowed node name and path lengths in Utils.java by getting an exception when attempting to create nodes which violated the policy. I believe this is a backwards compatibility issue and the following should occur: 1) there should be documentation about this. Why are there maximums? Can the default maximums be increased? Is this only applicable to some MKs like Mongo or are they enforced globally? 2) the values themselves should be OSGI properties, not System Properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2637) update Postgres JDBC version to 9.4
Julian Reschke created OAK-2637: --- Summary: update Postgres JDBC version to 9.4 Key: OAK-2637 URL: https://issues.apache.org/jira/browse/OAK-2637 Project: Jackrabbit Oak Issue Type: Sub-task Components: rdbmk Reporter: Julian Reschke Assignee: Julian Reschke Priority: Trivial Fix For: 1.0.13, 1.1.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2637) update Postgres JDBC version to 9.4
[ https://issues.apache.org/jira/browse/OAK-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-2637. - Resolution: Fixed update Postgres JDBC version to 9.4 --- Key: OAK-2637 URL: https://issues.apache.org/jira/browse/OAK-2637 Project: Jackrabbit Oak Issue Type: Sub-task Components: rdbmk Reporter: Julian Reschke Assignee: Julian Reschke Priority: Trivial Fix For: 1.0.13, 1.1.9 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2609) Thread.interrupt seems to stop repository
[ https://issues.apache.org/jira/browse/OAK-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363484#comment-14363484 ] Michael Dürig commented on OAK-2609: bq. Can this at least be catched somehow with a big log message? The problem is probably that there is no single place nor a single exception to catch. To cover them all we'd need to go through each and every place where something in Oak does a blocking operation and check the side effects. Maybe a good compromise is to come up with a test case provoking the problem and just cover the cases we see coming up with that assuming those will be the most frequent ones. Thread.interrupt seems to stop repository - Key: OAK-2609 URL: https://issues.apache.org/jira/browse/OAK-2609 Project: Jackrabbit Oak Issue Type: Bug Affects Versions: 1.1.7 Environment: Java 7 Reporter: Carsten Ziegeler Priority: Critical Fix For: 1.1.8, 1.0.13 We have a sporadic problem with Sling's JCR installer 3.3.8 and Oak (tar mk). It seems to timing related: the JCR installer does a Thread#interrupt at one point and sometimes this brings the hole instance to stop. Nothing else is going on any more. While of course, a workaround is to remove the Thread.interrupt call in the JCR installer (which we did, see SLING-4477), I have the fear that this can happen with any code that is using the repository and gets interrupted. This error is hard to reproduce, however with three people testing we could see this several times happening -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-2609) Thread.interrupt seems to stop repository
[ https://issues.apache.org/jira/browse/OAK-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363255#comment-14363255 ] Michael Dürig edited comment on OAK-2609 at 3/16/15 2:40 PM: - I don't think we should do anything here on the Oak side as we are dealing with [documented side effects|http://docs.oracle.com/javase/7/docs/api/java/nio/channels/InterruptibleChannel.html] of {{Thread.interrupt}}. We should rather discourage the usage of {{Thread.interrupt}} as: * it is overloaded in multiple unpredictable ways, * {{catch (InterruptedException ignore)}} is prevalent (and usually wrong), * in most cases it breaks separation of concerns (concurrency vs. business logic). We should instead provide guidance to users how to best replace usages of {{Thread.interrupt}} by safer alternatives. OTOH fixing this in Oak means concealing deeper issues and depending on your POV event breaking above general contract on channels. was (Author: mduerig): I don't think we should do anything here on the Oak side as we are dealing with [documented side effects|http://docs.oracle.com/javase/7/docs/api/java/nio/channels/InterruptibleChannel.html] of {{Thread.interrupt}}: Thread.interrupt seems to stop repository - Key: OAK-2609 URL: https://issues.apache.org/jira/browse/OAK-2609 Project: Jackrabbit Oak Issue Type: Bug Affects Versions: 1.1.7 Environment: Java 7 Reporter: Carsten Ziegeler Priority: Critical Fix For: 1.1.8, 1.0.13 We have a sporadic problem with Sling's JCR installer 3.3.8 and Oak (tar mk). It seems to timing related: the JCR installer does a Thread#interrupt at one point and sometimes this brings the hole instance to stop. Nothing else is going on any more. While of course, a workaround is to remove the Thread.interrupt call in the JCR installer (which we did, see SLING-4477), I have the fear that this can happen with any code that is using the repository and gets interrupted. This error is hard to reproduce, however with three people testing we could see this several times happening -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2635) TimeSeriesMax's frequent 'drops to 0'
Stefan Egli created OAK-2635: Summary: TimeSeriesMax's frequent 'drops to 0' Key: OAK-2635 URL: https://issues.apache.org/jira/browse/OAK-2635 Project: Jackrabbit Oak Issue Type: Bug Components: core Affects Versions: 1.0.12 Reporter: Stefan Egli The current implementation of TimeSeriesMax - which is what is backing eg the very important 'ObservationQueueMaxLength' statistics - has a very infamous behavior: it does very frequent, intermittent 'jumps back to 0'. This even though the queue-lengths are still at the previous highs, as can often be seen with subsequent measurements (which eg are still showing there are 1000 events in the observation queue). The reason seems to be that * the value is increased via {{TimeSeriesMax.recordValue()}} during a 1 second interval * reset to 0 via {{TimeSeriesMaxinit.run()}} every second So basically, every second the counter is reset, then during 1 second if any call to {{recordValue()}} happens, it is increased. This in my view is rather unfortunate - as it can result in mentioned 'jumpy-0' behavior, but it can also jump to values in between if the largest queue does not reports its length during 1 second. It sounds a bit like this was done this way intentionally? (perhaps to make it as inexpensive as possible) or could this be fixed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2609) Thread.interrupt seems to stop repository
[ https://issues.apache.org/jira/browse/OAK-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363343#comment-14363343 ] Carsten Ziegeler commented on OAK-2609: --- Just to be clear: I'm not encouraging the use of Thread.interrupt - in many cases it can be replaced by a better way. However, as noted, I've seen a Thread.interrupt call bringing down Oak and while I agree that replacing this with something else is a better solution, I'm not sure about how practible this is. If you own all the code in the world, you can do this. Can this at least be catched somehow with a big log message? Thread.interrupt seems to stop repository - Key: OAK-2609 URL: https://issues.apache.org/jira/browse/OAK-2609 Project: Jackrabbit Oak Issue Type: Bug Affects Versions: 1.1.7 Environment: Java 7 Reporter: Carsten Ziegeler Priority: Critical Fix For: 1.1.8, 1.0.13 We have a sporadic problem with Sling's JCR installer 3.3.8 and Oak (tar mk). It seems to timing related: the JCR installer does a Thread#interrupt at one point and sometimes this brings the hole instance to stop. Nothing else is going on any more. While of course, a workaround is to remove the Thread.interrupt call in the JCR installer (which we did, see SLING-4477), I have the fear that this can happen with any code that is using the repository and gets interrupted. This error is hard to reproduce, however with three people testing we could see this several times happening -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2072) Lucene: inconsistent usage of the config option persistence
[ https://issues.apache.org/jira/browse/OAK-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-2072: - Fix Version/s: (was: 1.2) 1.4 Lucene: inconsistent usage of the config option persistence - Key: OAK-2072 URL: https://issues.apache.org/jira/browse/OAK-2072 Project: Jackrabbit Oak Issue Type: Bug Components: query Reporter: Thomas Mueller Assignee: Alex Parvulescu Fix For: 1.4 The Lucene index reader uses the configuration property persistence, but the editor (the component updating the index) does not. That leads to very strange behavior if the property is missing, but the property file is set: the reader would try to read from the file system, but those files are not updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2476) Move our CI to Jenkins
[ https://issues.apache.org/jira/browse/OAK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363400#comment-14363400 ] Tommaso Teofili commented on OAK-2476: -- windows Jenkins slaves seem to be back again, triggering a new build to check that's the case. Move our CI to Jenkins -- Key: OAK-2476 URL: https://issues.apache.org/jira/browse/OAK-2476 Project: Jackrabbit Oak Issue Type: Task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Priority: Critical Labels: CI, build, infrastructure Fix For: 1.2 We should strive for stabilization of our CI setup, as of now we had Buildbot and Travis. It seems ASF Jenkins can perform jobs on different environments (*nix, Windows and others) so we can evaluate that and check if it better address our needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-2635) TimeSeriesMax's frequent 'drops to 0'
[ https://issues.apache.org/jira/browse/OAK-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig reassigned OAK-2635: -- Assignee: Michael Dürig TimeSeriesMax's frequent 'drops to 0' - Key: OAK-2635 URL: https://issues.apache.org/jira/browse/OAK-2635 Project: Jackrabbit Oak Issue Type: Bug Components: core Affects Versions: 1.0.12 Reporter: Stefan Egli Assignee: Michael Dürig The current implementation of TimeSeriesMax - which is what is backing eg the very important 'ObservationQueueMaxLength' statistics - has a very infamous behavior: it does very frequent, intermittent 'jumps back to 0'. This even though the queue-lengths are still at the previous highs, as can often be seen with subsequent measurements (which eg are still showing there are 1000 events in the observation queue). The reason seems to be that * the value is increased via {{TimeSeriesMax.recordValue()}} during a 1 second interval * reset to 0 via {{TimeSeriesMaxinit.run()}} every second So basically, every second the counter is reset, then during 1 second if any call to {{recordValue()}} happens, it is increased. This in my view is rather unfortunate - as it can result in mentioned 'jumpy-0' behavior, but it can also jump to values in between if the largest queue does not reports its length during 1 second. It sounds a bit like this was done this way intentionally? (perhaps to make it as inexpensive as possible) or could this be fixed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2635) TimeSeriesMax's frequent 'drops to 0'
[ https://issues.apache.org/jira/browse/OAK-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2635: --- Fix Version/s: 1.2 TimeSeriesMax's frequent 'drops to 0' - Key: OAK-2635 URL: https://issues.apache.org/jira/browse/OAK-2635 Project: Jackrabbit Oak Issue Type: Bug Components: core Affects Versions: 1.0.12 Reporter: Stefan Egli Assignee: Michael Dürig Fix For: 1.2 The current implementation of TimeSeriesMax - which is what is backing eg the very important 'ObservationQueueMaxLength' statistics - has a very infamous behavior: it does very frequent, intermittent 'jumps back to 0'. This even though the queue-lengths are still at the previous highs, as can often be seen with subsequent measurements (which eg are still showing there are 1000 events in the observation queue). The reason seems to be that * the value is increased via {{TimeSeriesMax.recordValue()}} during a 1 second interval * reset to 0 via {{TimeSeriesMaxinit.run()}} every second So basically, every second the counter is reset, then during 1 second if any call to {{recordValue()}} happens, it is increased. This in my view is rather unfortunate - as it can result in mentioned 'jumpy-0' behavior, but it can also jump to values in between if the largest queue does not reports its length during 1 second. It sounds a bit like this was done this way intentionally? (perhaps to make it as inexpensive as possible) or could this be fixed? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1859) Migration from TarMK to MongoMK
[ https://issues.apache.org/jira/browse/OAK-1859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363941#comment-14363941 ] Przemo Pakulski commented on OAK-1859: -- Question: Does it rely on backup/restore functionality ? Or is it (will be) exposed as separate oak-run command? Migration from TarMK to MongoMK --- Key: OAK-1859 URL: https://issues.apache.org/jira/browse/OAK-1859 Project: Jackrabbit Oak Issue Type: Wish Components: upgrade Affects Versions: 1.0 Reporter: Przemo Pakulski Assignee: Manfred Baedke Fix For: 1.1.8, 1.0.13 Is migration from TarMK to MongoMK possible ? Let's say I initially used TarMK, then realized that Mongo is better option, is there any way to migrate existing content to MongoDB ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1648) Creating multiple checkpoint on same head revision overwrites previous entries
[ https://issues.apache.org/jira/browse/OAK-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-1648: -- Fix Version/s: (was: 1.2) 1.3.0 Moved to 1.3 because this is only a minor issue and we currently only have a single client for the checkpoint functionality. The async index update always created a checkpoint with a fixed time-to-live. An optimistic update in a cluster is still possible, even without the _modCount. The update would simply include a check on the expiryTime. Creating multiple checkpoint on same head revision overwrites previous entries -- Key: OAK-1648 URL: https://issues.apache.org/jira/browse/OAK-1648 Project: Jackrabbit Oak Issue Type: Bug Components: mongomk Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.3.0 Currently when a checkpoint is created in DocumentNodeStore then it is saved in form of currentHeadRev=expiryTime. Now if multiple checkpoints are created where head revision has not changed then only the last one would be saved and previous entries would be overridden as revision is used as key One fix would be to change the expiry time only if the new expiry time is greater than previous entry. However doing that safely in a cluster (check then save) is currently not possible with DocumentStore API as the modCount check if only supported for Nodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2499) Expose mongo and db versions for reporting purposes
[ https://issues.apache.org/jira/browse/OAK-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362842#comment-14362842 ] Marcel Reutegger commented on OAK-2499: --- I think it's fine to merge. Expose mongo and db versions for reporting purposes --- Key: OAK-2499 URL: https://issues.apache.org/jira/browse/OAK-2499 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk, rdbmk, segmentmk Reporter: Marius Petria Assignee: Chetan Mehrotra Fix For: 1.1.8 Attachments: OAK-2499.1.diff, OAK-2499.2.diff For reporting purposes I need to find out from java the MK type and also for mongo version and the info about used DB make/version? [~chetanm] suggested that such information can be exposed when we register a {{NodeStore}} instances within OSGi we can also add a service property which captures such implementation details. Possibly use service.description or define a new one which provide a comma separated list of attributes.” -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2392) [DocumentMK] Garbage Collect older revisions of binary properties in main document
[ https://issues.apache.org/jira/browse/OAK-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362900#comment-14362900 ] Marcel Reutegger commented on OAK-2392: --- I think it's better to implement this as part of the revision GC. This way, the blob GC can focus on cleaning up the blobs only. Linked this issue to OAK-1981. [DocumentMK] Garbage Collect older revisions of binary properties in main document -- Key: OAK-2392 URL: https://issues.apache.org/jira/browse/OAK-2392 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.2 Current GC logic for DocumentMK only collects certain types of garbage (see OAK-1981) and currently only split documents are removed. While complete full blow gc would take time and yet not fully implemented we should handle those documents which have binary properties and those properties get updated few times (but not very frequently). For e.g. performing a reindex for Lucene index would lead to removal of index files nodes and again creation of nodes with same name. In such a case the older revision of binary property would remain in main document and would not be eligible for gc as per current impl. As a fix the GC logic should look for document which might have binaries and then remove the older revisions of binary properties. Currently we do scan all such documents for Blob GC. So this can be done either as part of Revision GC or Blob GC -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2392) [DocumentMK] Garbage Collect older revisions of binary properties in main document
[ https://issues.apache.org/jira/browse/OAK-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362901#comment-14362901 ] Marcel Reutegger commented on OAK-2392: --- I don't consider this a blocker for 1.2. [~chetanm], should we move it to 1.3 and backport it later if necessary? [DocumentMK] Garbage Collect older revisions of binary properties in main document -- Key: OAK-2392 URL: https://issues.apache.org/jira/browse/OAK-2392 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.2 Current GC logic for DocumentMK only collects certain types of garbage (see OAK-1981) and currently only split documents are removed. While complete full blow gc would take time and yet not fully implemented we should handle those documents which have binary properties and those properties get updated few times (but not very frequently). For e.g. performing a reindex for Lucene index would lead to removal of index files nodes and again creation of nodes with same name. In such a case the older revision of binary property would remain in main document and would not be eligible for gc as per current impl. As a fix the GC logic should look for document which might have binaries and then remove the older revisions of binary properties. Currently we do scan all such documents for Blob GC. So this can be done either as part of Revision GC or Blob GC -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2624) Move out non essential task processing from DoucmentNodeStore background thread
[ https://issues.apache.org/jira/browse/OAK-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2624: -- Fix Version/s: (was: 1.2) 1.1.8 Move out non essential task processing from DoucmentNodeStore background thread --- Key: OAK-2624 URL: https://issues.apache.org/jira/browse/OAK-2624 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Chetan Mehrotra Assignee: Marcel Reutegger Fix For: 1.1.8 The {{DocumentNodeStore background thread}} handles quite a few task currently like # Clean orphaned branches # Clean collisions # Split documents # Background write - write back pending updates to _lastRev # Detect change in head revision from backend # Cache invalidation # Revision Comparator purge The background operation is scheduled to be performed every second (1s frequency). However at times some of the above task take more time which delay in picking up the changes from backend. For quick propagation of changes in cluster it is essential that work done in background thread is performed quickly. To enable that we should move out task which are not directly related to _head revision change_ like splitDocument. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-2624) Move out non essential task processing from DoucmentNodeStore background thread
[ https://issues.apache.org/jira/browse/OAK-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger reassigned OAK-2624: - Assignee: Marcel Reutegger Move out non essential task processing from DoucmentNodeStore background thread --- Key: OAK-2624 URL: https://issues.apache.org/jira/browse/OAK-2624 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Chetan Mehrotra Assignee: Marcel Reutegger Fix For: 1.2 The {{DocumentNodeStore background thread}} handles quite a few task currently like # Clean orphaned branches # Clean collisions # Split documents # Background write - write back pending updates to _lastRev # Detect change in head revision from backend # Cache invalidation # Revision Comparator purge The background operation is scheduled to be performed every second (1s frequency). However at times some of the above task take more time which delay in picking up the changes from backend. For quick propagation of changes in cluster it is essential that work done in background thread is performed quickly. To enable that we should move out task which are not directly related to _head revision change_ like splitDocument. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2634) QueryEngine should expose name query as property restriction
Chetan Mehrotra created OAK-2634: Summary: QueryEngine should expose name query as property restriction Key: OAK-2634 URL: https://issues.apache.org/jira/browse/OAK-2634 Project: Jackrabbit Oak Issue Type: Sub-task Components: query Reporter: Chetan Mehrotra Fix For: 1.2 Currently {{NodeNameImpl}} and {{NodeLocalNameImpl}} do not add restriction to a filter hence query index cannot handle such queries. To allow faster execution name related restriction can be converted to a property restriction -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1963) Expose file system path of Blob
[ https://issues.apache.org/jira/browse/OAK-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363068#comment-14363068 ] Michael Dürig commented on OAK-1963: I meant the discussion around and the concerns brought forward in JCR-3534. In a way that issue is a special case of this one and we should probably look into generalising the solution accordingly. Expose file system path of Blob --- Key: OAK-1963 URL: https://issues.apache.org/jira/browse/OAK-1963 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Pralaypati Ta Assignee: Chetan Mehrotra Fix For: 1.2 In some situations direct file system path is more useful than repository path e.g. native tools don't understand repository path, instead file system path can be passed directly to native tools for processing binary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2476) Move our CI to Jenkins
[ https://issues.apache.org/jira/browse/OAK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363076#comment-14363076 ] Michael Dürig commented on OAK-2476: I see PermGen space error for the pedantic builds now. Might this have to do with findbugs? Can we just increase the perm gen size for that build? Move our CI to Jenkins -- Key: OAK-2476 URL: https://issues.apache.org/jira/browse/OAK-2476 Project: Jackrabbit Oak Issue Type: Task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Priority: Critical Labels: CI, build, infrastructure Fix For: 1.2 We should strive for stabilization of our CI setup, as of now we had Buildbot and Travis. It seems ASF Jenkins can perform jobs on different environments (*nix, Windows and others) so we can evaluate that and check if it better address our needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-2476) Move our CI to Jenkins
[ https://issues.apache.org/jira/browse/OAK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363076#comment-14363076 ] Michael Dürig edited comment on OAK-2476 at 3/16/15 11:14 AM: -- I see PermGen space error for the pedantic builds now [1]. Might this have to do with findbugs? Can we just increase the perm gen size for that build? [1] https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/15/jdk=latest1.6,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=pedantic/console was (Author: mduerig): I see PermGen space error for the pedantic builds now. Might this have to do with findbugs? Can we just increase the perm gen size for that build? Move our CI to Jenkins -- Key: OAK-2476 URL: https://issues.apache.org/jira/browse/OAK-2476 Project: Jackrabbit Oak Issue Type: Task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Priority: Critical Labels: CI, build, infrastructure Fix For: 1.2 We should strive for stabilization of our CI setup, as of now we had Buildbot and Travis. It seems ASF Jenkins can perform jobs on different environments (*nix, Windows and others) so we can evaluate that and check if it better address our needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2476) Move our CI to Jenkins
[ https://issues.apache.org/jira/browse/OAK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363086#comment-14363086 ] Tommaso Teofili commented on OAK-2476: -- actually I used the wrong argument as the problem is in the PermGen. However thinking more to it I prefer to make the pedantic build run only on JDK 8 which got rid of Heap/PermGen limitations and in fact it doesn't have such a problem. Move our CI to Jenkins -- Key: OAK-2476 URL: https://issues.apache.org/jira/browse/OAK-2476 Project: Jackrabbit Oak Issue Type: Task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Priority: Critical Labels: CI, build, infrastructure Fix For: 1.2 We should strive for stabilization of our CI setup, as of now we had Buildbot and Travis. It seems ASF Jenkins can perform jobs on different environments (*nix, Windows and others) so we can evaluate that and check if it better address our needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2392) [DocumentMK] Garbage Collect older revisions of binary properties in main document
[ https://issues.apache.org/jira/browse/OAK-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-2392: - Fix Version/s: (was: 1.2) 1.3.1 [DocumentMK] Garbage Collect older revisions of binary properties in main document -- Key: OAK-2392 URL: https://issues.apache.org/jira/browse/OAK-2392 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.3.1 Current GC logic for DocumentMK only collects certain types of garbage (see OAK-1981) and currently only split documents are removed. While complete full blow gc would take time and yet not fully implemented we should handle those documents which have binary properties and those properties get updated few times (but not very frequently). For e.g. performing a reindex for Lucene index would lead to removal of index files nodes and again creation of nodes with same name. In such a case the older revision of binary property would remain in main document and would not be eligible for gc as per current impl. As a fix the GC logic should look for document which might have binaries and then remove the older revisions of binary properties. Currently we do scan all such documents for Blob GC. So this can be done either as part of Revision GC or Blob GC -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2263) Avoid unnecessary reads on index lookup
[ https://issues.apache.org/jira/browse/OAK-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-2263. --- Resolution: Duplicate Avoid unnecessary reads on index lookup --- Key: OAK-2263 URL: https://issues.apache.org/jira/browse/OAK-2263 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor The PropertyIndex looks up the matching index definitions multiple times. This means each time all definitions are read to find out which one matches the given property restriction. AFAICS this happens at least six times. The first two times when the cost is calculated: 1) check if the property is indexed and 2) calculate the cost. The next four times when the query is executed: 3) check if the property is indexed, 4) calculate the cost, 5) check if the property is indexed, 6) lookup index to create cursor for query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2263) Avoid unnecessary reads on index lookup
[ https://issues.apache.org/jira/browse/OAK-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362910#comment-14362910 ] Marcel Reutegger commented on OAK-2263: --- Already reported as OAK-1273. Resolving this issue as a duplicate. Avoid unnecessary reads on index lookup --- Key: OAK-2263 URL: https://issues.apache.org/jira/browse/OAK-2263 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor The PropertyIndex looks up the matching index definitions multiple times. This means each time all definitions are read to find out which one matches the given property restriction. AFAICS this happens at least six times. The first two times when the cost is calculated: 1) check if the property is indexed and 2) calculate the cost. The next four times when the query is executed: 3) check if the property is indexed, 4) calculate the cost, 5) check if the property is indexed, 6) lookup index to create cursor for query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1591) org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT fails
[ https://issues.apache.org/jira/browse/OAK-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-1591: -- Fix Version/s: (was: 1.2) 1.3.0 [~reschke], do you still see those failures? org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT fails -- Key: OAK-1591 URL: https://issues.apache.org/jira/browse/OAK-1591 Project: Jackrabbit Oak Issue Type: Task Components: mongomk Reporter: Julian Reschke Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.3.0 Fails frequently on my W7 desktop: testCacheInvalidationHierarchicalNotExist(org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT) Time elapsed: 0.04 sec FAILURE! java.lang.AssertionError at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT.testCacheInvalidationHierarchicalNotExist(CacheInvalidationIT.java:171) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2263) Avoid unnecessary reads on index lookup
[ https://issues.apache.org/jira/browse/OAK-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2263: -- Fix Version/s: (was: 1.4) Avoid unnecessary reads on index lookup --- Key: OAK-2263 URL: https://issues.apache.org/jira/browse/OAK-2263 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor The PropertyIndex looks up the matching index definitions multiple times. This means each time all definitions are read to find out which one matches the given property restriction. AFAICS this happens at least six times. The first two times when the cost is calculated: 1) check if the property is indexed and 2) calculate the cost. The next four times when the query is executed: 3) check if the property is indexed, 4) calculate the cost, 5) check if the property is indexed, 6) lookup index to create cursor for query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1589) MongoDocumentStore fails to report error for keys that are too long
[ https://issues.apache.org/jira/browse/OAK-1589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-1589: -- Fix Version/s: (was: 1.2) 1.3.0 Postponing to 1.3 since Oak requires MongoDB 2.6.x anyway. MongoDocumentStore fails to report error for keys that are too long --- Key: OAK-1589 URL: https://issues.apache.org/jira/browse/OAK-1589 Project: Jackrabbit Oak Issue Type: Bug Components: mongomk Reporter: Julian Reschke Fix For: 1.3.0 The MongoDocumentStore fails to report an error when the key length exceeds the allowable width in MongoDB. This can be fixed by using a newer version of MongoDB (mongodb 2.5.5 -- see https://jira.mongodb.org/browse/SERVER-5290) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1974) Fail fast on branch conflict
[ https://issues.apache.org/jira/browse/OAK-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-1974: -- Fix Version/s: (was: 1.2) 1.3.0 Fail fast on branch conflict Key: OAK-1974 URL: https://issues.apache.org/jira/browse/OAK-1974 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Fix For: 1.3.0 The current MongoMK implementation performs retries when it runs into merge conflicts caused by collisions. It may be possible to resolve a conflict by resetting the branch back to the state as it was before the merge and re-run the commit hooks again. This helps if the conflict was introduced by a commit hook. At the moment the retries also happen when the conflict was introduced before the merge. In this case, a retry is useless and the commit should fail fast. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1860) unit tests for concurrent DocumentStore access
[ https://issues.apache.org/jira/browse/OAK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-1860: -- Fix Version/s: (was: 1.2) 1.3.0 unit tests for concurrent DocumentStore access -- Key: OAK-1860 URL: https://issues.apache.org/jira/browse/OAK-1860 Project: Jackrabbit Oak Issue Type: Sub-task Components: mongomk Affects Versions: 1.1.0 Reporter: Julian Reschke Fix For: 1.3.0 Add tests that exercise multiple DS instances accessing the same persistence. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2106) Optimize reads from secondaries
[ https://issues.apache.org/jira/browse/OAK-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2106: -- Fix Version/s: (was: 1.2) 1.3.0 Optimize reads from secondaries --- Key: OAK-2106 URL: https://issues.apache.org/jira/browse/OAK-2106 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Fix For: 1.3.0 OAK-1645 introduced support for reads from secondaries under certain conditions. The current implementation checks the _lastRev on a potentially cached parent document and reads from a secondary if it has not been modified in the last 24 hours. This timespan is somewhat arbitrary but reflects the assumption that the replication lag of a secondary shouldn't be more than 24 hours. This logic should be optimized to take the actual replication lag into account. MongoDB provides information about the replication lag with the command rs.status(). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2574) Update mongo-java-driver to 2.13.0
[ https://issues.apache.org/jira/browse/OAK-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2574: -- Fix Version/s: (was: 1.2) 1.1.8 I think we should do this before the 1.2 release. Update mongo-java-driver to 2.13.0 -- Key: OAK-2574 URL: https://issues.apache.org/jira/browse/OAK-2574 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Fix For: 1.1.8 MongoDB 3.0 was released yesterday and the current java driver (2.12.2) used by Oak is marked as untested with 3.0: http://docs.mongodb.org/ecosystem/drivers/java/#compatibility We should update to 2.13.0, which is compatible with 3.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1322) Reduce calls to MongoDB
[ https://issues.apache.org/jira/browse/OAK-1322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-1322: -- Fix Version/s: (was: 1.2) 1.3.0 Reduce calls to MongoDB --- Key: OAK-1322 URL: https://issues.apache.org/jira/browse/OAK-1322 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Fix For: 1.3.0 Attachments: OAK-1322-mreutegg.patch As discussed with Chetan offline we'd like to reduce the number of calls to MongoDB when content is added to the repository with a filevault package import. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2108) Killing a cluster node may stop async index update to to 15 minutes
[ https://issues.apache.org/jira/browse/OAK-2108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2108: -- Issue Type: Improvement (was: Bug) Killing a cluster node may stop async index update to to 15 minutes --- Key: OAK-2108 URL: https://issues.apache.org/jira/browse/OAK-2108 Project: Jackrabbit Oak Issue Type: Improvement Components: query Reporter: Thomas Mueller Fix For: 1.2 When killing a cluster node that is running the sync index update, then this async index update will not run for up to 15 minutes, because the lease time is set to 15 minutes. I think the lease time should be much smaller, for example 1 minute, or maybe even 10 seconds. Also, we might need to better document this issue (in addition to the warning in the log file). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-114) MicroKernel API: specify retention policy for old revisions
[ https://issues.apache.org/jira/browse/OAK-114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-114: - Fix Version/s: (was: 1.2) MicroKernel API: specify retention policy for old revisions --- Key: OAK-114 URL: https://issues.apache.org/jira/browse/OAK-114 Project: Jackrabbit Oak Issue Type: Improvement Components: mk Reporter: Stefan Guggisberg Assignee: Stefan Guggisberg Labels: api Attachments: OAK-114.patch the MicroKernel API javadoc should specify the minimal guaranteed retention period for old revisions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2108) Killing a cluster node may stop async index update to to 15 minutes
[ https://issues.apache.org/jira/browse/OAK-2108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2108: -- Fix Version/s: (was: 1.2) 1.4 Killing a cluster node may stop async index update to to 15 minutes --- Key: OAK-2108 URL: https://issues.apache.org/jira/browse/OAK-2108 Project: Jackrabbit Oak Issue Type: Improvement Components: query Reporter: Thomas Mueller Fix For: 1.4 When killing a cluster node that is running the sync index update, then this async index update will not run for up to 15 minutes, because the lease time is set to 15 minutes. I think the lease time should be much smaller, for example 1 minute, or maybe even 10 seconds. Also, we might need to better document this issue (in addition to the warning in the log file). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2613) Do versionGC more frequently and at adjusted speed
[ https://issues.apache.org/jira/browse/OAK-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-2613: - Fix Version/s: (was: 1.2) 1.3.0 Flagged this 1.3 instead of 1.2 as this is not a blocker for most apps. Do versionGC more frequently and at adjusted speed -- Key: OAK-2613 URL: https://issues.apache.org/jira/browse/OAK-2613 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Affects Versions: 1.0.12 Reporter: Stefan Egli Fix For: 1.3.0 This is a follow-up ticket from [here|https://issues.apache.org/jira/browse/OAK-2557?focusedCommentId=14355322page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14355322] mixed with an offline discussion with [~mreutegg]: * we could make the VersionGC play nicely to existing load on the system: it could progress slower if the load is higher and vice-verca. One simple measure could be: if the observation queue is small (eg below 10) then the load is low and it could progress full-speed. Otherwise it could add some artificial sleeping in between. * we could run versionGC more regularly than once a day but instead kind of 'continuously' let it run in the background. While the speed of the gc would be adjusted to the load - it also would have to be assured that it doesn't run too slow (and would never finish if instance is under some constant load) Note that 'adjusted speed' would also imply some intelligence about the system load, as pointed out by [~chetanm] on OAK-2557: {quote}Version GC currently ensures that query fired is made against the Secondary (if present). However having some throttling in such background task would be good thing to have. But first we need to have some SystemLoadIndicator notion in Oak which can be provide details say in percentage 1..100 about system load. We can then expose configurable threshold which VersionGC would listen for and adjust its working accordingly. It can be a JMX bean which emits notification and we have our components listen to those notification (or use OSGi SR/Events). That can be used in other places like Observation processing, Blob GC etc {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1786) ExcerptTest#testMatchMultipleNonMatchingTokens is failing
[ https://issues.apache.org/jira/browse/OAK-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-1786: - Fix Version/s: 1.2 ExcerptTest#testMatchMultipleNonMatchingTokens is failing - Key: OAK-1786 URL: https://issues.apache.org/jira/browse/OAK-1786 Project: Jackrabbit Oak Issue Type: Bug Components: oak-solr Affects Versions: 1.1.0 Reporter: Alex Parvulescu Fix For: 1.2 Trace: {code} testMatchMultipleNonMatchingTokens(org.apache.jackrabbit.core.query.ExcerptTest) Time elapsed: 0.163 sec FAILURE! junit.framework.AssertionFailedError: expected:1 but was:0 at junit.framework.Assert.fail(Assert.java:50) at junit.framework.Assert.failNotEquals(Assert.java:287) at junit.framework.Assert.assertEquals(Assert.java:67) at junit.framework.Assert.assertEquals(Assert.java:134) at junit.framework.Assert.assertEquals(Assert.java:140) at org.apache.jackrabbit.core.query.ExcerptTest.testMatchMultipleNonMatchingTokens(ExcerptTest.java:333) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1611) Support also LBHttpSolrServer in RemoteSolrServerProvider
[ https://issues.apache.org/jira/browse/OAK-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-1611: - Fix Version/s: 1.4 Support also LBHttpSolrServer in RemoteSolrServerProvider - Key: OAK-1611 URL: https://issues.apache.org/jira/browse/OAK-1611 Project: Jackrabbit Oak Issue Type: Improvement Components: oak-solr Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.4 Solr's LBHttpSolrServer can do software load balancing between Solr instances (should be all replicas with the same underlying index to work here) and it'd be good to provide support for that in RemoteSolrServerProvider. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-653) Improve binaries handling in Solr index
[ https://issues.apache.org/jira/browse/OAK-653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-653: Fix Version/s: 1.2 Improve binaries handling in Solr index --- Key: OAK-653 URL: https://issues.apache.org/jira/browse/OAK-653 Project: Jackrabbit Oak Issue Type: Improvement Components: oak-solr Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.2 Solr provides SolrCell (integration with Apache Tika, http://wiki.apache.org/solr/ExtractingRequestHandler) which would be easy to leverage. Also it'd be nice to have that working on the Lucene level as a specific set of analyzers/tokenizers so that it'd be transparent (wouldn't need any special URI for binaries indexing) once those are configured in a Solr schema. It'd be also good to be able to extract the text from within the SolrIndexEditor (like LuceneIndexEditor does) without having to rely on SolrCell on the Solr side as it's not always exposed (it depends on wether it's explicitly configured). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1610) Improved default indexing by JCR type in SolrIndexEditor
[ https://issues.apache.org/jira/browse/OAK-1610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-1610: - Fix Version/s: 1.2 Improved default indexing by JCR type in SolrIndexEditor Key: OAK-1610 URL: https://issues.apache.org/jira/browse/OAK-1610 Project: Jackrabbit Oak Issue Type: Improvement Components: oak-solr Reporter: Tommaso Teofili Fix For: 1.2 It'd be good to provide a typed indexing default so that properties of a certain type can be mapped to certain Solr dynamic fields with dedicated types. The infrastructure for doing that is already in place as per OakSolrConfiguration#getFieldNameFor(Type) but the default configuration is not properly set with a good mapping. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1686) failing test org.apache.jackrabbit.core.query.VersionStoreQueryTest.testCheckin
[ https://issues.apache.org/jira/browse/OAK-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-1686: - Fix Version/s: 1.2 failing test org.apache.jackrabbit.core.query.VersionStoreQueryTest.testCheckin --- Key: OAK-1686 URL: https://issues.apache.org/jira/browse/OAK-1686 Project: Jackrabbit Oak Issue Type: Bug Components: oak-solr Affects Versions: 0.20 Reporter: Julian Reschke Assignee: Tommaso Teofili Fix For: 1.2 Fails with: Failed tests: testCheckin(org.apache.jackrabbit.core.query.VersionStoreQueryTest): /jcr:system/jcr:versionStorage/63/8e/9b/638e9bb0-e307-4402-8b8a-4bb0be52eec9/1.1/jcr:frozenNode is not part of the result set [/testroot/node1, /jcr:system/jcr:versionStorage/63/8e/9b/638e9bb0-e307-4402-8b8a-4bb0be52eec9/1.0/jcr:frozenNode] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2499) Expose mongo and db versions for reporting purposes
[ https://issues.apache.org/jira/browse/OAK-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-2499. -- Resolution: Fixed Issues is fixed on trunk. Would still need to be ported to branch Expose mongo and db versions for reporting purposes --- Key: OAK-2499 URL: https://issues.apache.org/jira/browse/OAK-2499 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk, rdbmk, segmentmk Reporter: Marius Petria Assignee: Chetan Mehrotra Fix For: 1.1.8 Attachments: OAK-2499.1.diff, OAK-2499.2.diff For reporting purposes I need to find out from java the MK type and also for mongo version and the info about used DB make/version? [~chetanm] suggested that such information can be exposed when we register a {{NodeStore}} instances within OSGi we can also add a service property which captures such implementation details. Possibly use service.description or define a new one which provide a comma separated list of attributes.” -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2374) Sporadic test failure of OSGiIT.listBundles on Buildbot
[ https://issues.apache.org/jira/browse/OAK-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-2374: - Fix Version/s: (was: 1.1.8) Sporadic test failure of OSGiIT.listBundles on Buildbot --- Key: OAK-2374 URL: https://issues.apache.org/jira/browse/OAK-2374 Project: Jackrabbit Oak Issue Type: Bug Components: it Reporter: Michael Dürig Assignee: Chetan Mehrotra Priority: Minor Labels: CI, buildbot, test Fix For: 1.2 Attachments: OAK-2374.patch See http://markmail.org/message/idx2y2dwpkaxchsp for previous mention. I suggest to use the mechanism from OAK-2371 to exclude the tests on that CI environment for now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-2016) Make blob gc max age configurable in SegmentNodeStoreService
[ https://issues.apache.org/jira/browse/OAK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain reassigned OAK-2016: -- Assignee: Amit Jain Make blob gc max age configurable in SegmentNodeStoreService Key: OAK-2016 URL: https://issues.apache.org/jira/browse/OAK-2016 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Amit Jain Assignee: Amit Jain Priority: Minor Attachments: OAK-2016.patch The blob gc max age setting is not configurable when using {{SegmentNodeStoreService}}. This can be made configurable and will be useful for testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2629) Cleanup Oak Travis jobs
[ https://issues.apache.org/jira/browse/OAK-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362941#comment-14362941 ] Tommaso Teofili commented on OAK-2629: -- good point Marcel, I agree. After we get windows builds back to work that's the next thing in line I plan to have a look, I won't touch the current stuff until we have at least the same coverage on Jenkins. Cleanup Oak Travis jobs --- Key: OAK-2629 URL: https://issues.apache.org/jira/browse/OAK-2629 Project: Jackrabbit Oak Issue Type: Sub-task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.1.8, 1.2 Since we're moving toward Jenkins, let's remove the Travis jobs for Oak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2154) Support maven-javadoc-plugin -Xdoclint:none option for JDK 8
[ https://issues.apache.org/jira/browse/OAK-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-2154: --- Fix Version/s: (was: 1.2) Support maven-javadoc-plugin -Xdoclint:none option for JDK 8 Key: OAK-2154 URL: https://issues.apache.org/jira/browse/OAK-2154 Project: Jackrabbit Oak Issue Type: Bug Reporter: Amit Jain -Xdoclint:none option in maven-javdoc-plugin is only recognized in jdk 8 and hence build fails when generating javadocs with jdk 7. It should not be configured by default. One option is to create a new profile to enable builds with jdk 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-2154) Support maven-javadoc-plugin -Xdoclint:none option for JDK 8
[ https://issues.apache.org/jira/browse/OAK-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain reassigned OAK-2154: -- Assignee: Amit Jain Support maven-javadoc-plugin -Xdoclint:none option for JDK 8 Key: OAK-2154 URL: https://issues.apache.org/jira/browse/OAK-2154 Project: Jackrabbit Oak Issue Type: Bug Reporter: Amit Jain Assignee: Amit Jain -Xdoclint:none option in maven-javdoc-plugin is only recognized in jdk 8 and hence build fails when generating javadocs with jdk 7. It should not be configured by default. One option is to create a new profile to enable builds with jdk 8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2596) more (jmx) instrumentation for observation queue
[ https://issues.apache.org/jira/browse/OAK-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2596: --- Fix Version/s: (was: 1.1.9) 1.1.8 more (jmx) instrumentation for observation queue Key: OAK-2596 URL: https://issues.apache.org/jira/browse/OAK-2596 Project: Jackrabbit Oak Issue Type: Improvement Components: core Affects Versions: 1.0.12 Reporter: Stefan Egli Assignee: Michael Dürig Labels: monitoring, observation Fix For: 1.1.8 While debugging issues with the observation queue it would be handy to have more detailed information available. At the moment you can only see one value wrt length of the queue: that is the maximum of all queues. It is unclear if the queue is that long for only one or many listeners. And it is unclear from that if the listener is slow or the engine that produces the events for the listener. So I'd suggest to add the following details - possible exposed via JMX? : # add queue length details to each of the observation listeners # have a history of the last, eg 1000 events per listener showing a) how long the event took to be created/generated and b) how long the listener took to process. Sometimes averages are not detailed enough so such a in-depth information might become useful. (Not sure about the feasibility of '1000' here - maybe that could be configurable though - just putting the idea out here). # have some information about whether a listener is currently 'reading events from the cache' or whether it has to go to eg mongo # maybe have a 'top 10' listeners that have the largest queue at the moment to easily allow navigation instead of having to go through all (eg 200) listeners manually each time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-2632) Upgrade Jackrabbit dependency to 2.9.2
Michael Dürig created OAK-2632: -- Summary: Upgrade Jackrabbit dependency to 2.9.2 Key: OAK-2632 URL: https://issues.apache.org/jira/browse/OAK-2632 Project: Jackrabbit Oak Issue Type: Task Reporter: Michael Dürig Fix For: 1.1.8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2392) [DocumentMK] Garbage Collect older revisions of binary properties in main document
[ https://issues.apache.org/jira/browse/OAK-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362909#comment-14362909 ] Chetan Mehrotra commented on OAK-2392: -- Make sense. Moved to 1.3.1 [DocumentMK] Garbage Collect older revisions of binary properties in main document -- Key: OAK-2392 URL: https://issues.apache.org/jira/browse/OAK-2392 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.3.1 Current GC logic for DocumentMK only collects certain types of garbage (see OAK-1981) and currently only split documents are removed. While complete full blow gc would take time and yet not fully implemented we should handle those documents which have binary properties and those properties get updated few times (but not very frequently). For e.g. performing a reindex for Lucene index would lead to removal of index files nodes and again creation of nodes with same name. In such a case the older revision of binary property would remain in main document and would not be eligible for gc as per current impl. As a fix the GC logic should look for document which might have binaries and then remove the older revisions of binary properties. Currently we do scan all such documents for Blob GC. So this can be done either as part of Revision GC or Blob GC -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2575) improve documentation for DocumentStore.invalidateCache
[ https://issues.apache.org/jira/browse/OAK-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2575: -- Fix Version/s: (was: 1.2) 1.1.8 improve documentation for DocumentStore.invalidateCache --- Key: OAK-2575 URL: https://issues.apache.org/jira/browse/OAK-2575 Project: Jackrabbit Oak Issue Type: Task Components: mongomk Affects Versions: 1.2 Reporter: Julian Reschke Assignee: Marcel Reutegger Priority: Minor Fix For: 1.1.8 The JavaDoc currently just says invalidate the document cache; it would be good to clarify - that it's ok to keep cache entries as long as they get re-validated before being used and - whether it's ok to do the re-validation lazily (as opposed to immediately, as done by MongoDS) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1971) generalize MongoDB-specific tests
[ https://issues.apache.org/jira/browse/OAK-1971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-1971: -- Fix Version/s: (was: 1.2) 1.3.0 generalize MongoDB-specific tests - Key: OAK-1971 URL: https://issues.apache.org/jira/browse/OAK-1971 Project: Jackrabbit Oak Issue Type: Sub-task Components: mongomk Reporter: Julian Reschke Fix For: 1.3.0 It appears that there are tests that have been rewritten exclusively for the Mongo persistence, but which really are specific to the DocumentMK (formerly MongoMK). These should be rewritten to be useable for all implementations of the DocumentStore API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2420) DocumentNodeStore revision GC may lead to NPE
[ https://issues.apache.org/jira/browse/OAK-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2420: -- Fix Version/s: (was: 1.2) 1.1.8 DocumentNodeStore revision GC may lead to NPE - Key: OAK-2420 URL: https://issues.apache.org/jira/browse/OAK-2420 Project: Jackrabbit Oak Issue Type: Bug Components: core Affects Versions: 1.0 Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Fix For: 1.1.8 The DocumentNodeStore revision GC may cause a NPE in a reader thread when the GC deletes documents currently accessed by the reader. The {{docChildrenCache}} is invalidated in {{VersionGarbageCollector.collectDeletedDocuments()}} after documents are removed in the DocumentStore. The NPE may occur if removed documents are access in between. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1695) Document Solr index
[ https://issues.apache.org/jira/browse/OAK-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-1695: - Fix Version/s: 1.2 Document Solr index --- Key: OAK-1695 URL: https://issues.apache.org/jira/browse/OAK-1695 Project: Jackrabbit Oak Issue Type: Task Components: oak-solr Reporter: Tommaso Teofili Assignee: Tommaso Teofili Labels: documentation Fix For: 1.2 Provide documentation about the Oak Solr index. That should contain information about the design and how to configure it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2575) improve documentation for DocumentStore.invalidateCache
[ https://issues.apache.org/jira/browse/OAK-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-2575: -- Fix Version/s: (was: 1.1.8) 1.3.0 improve documentation for DocumentStore.invalidateCache --- Key: OAK-2575 URL: https://issues.apache.org/jira/browse/OAK-2575 Project: Jackrabbit Oak Issue Type: Task Components: mongomk Affects Versions: 1.2 Reporter: Julian Reschke Assignee: Marcel Reutegger Priority: Minor Fix For: 1.3.0 The JavaDoc currently just says invalidate the document cache; it would be good to clarify - that it's ok to keep cache entries as long as they get re-validated before being used and - whether it's ok to do the re-validation lazily (as opposed to immediately, as done by MongoDS) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1819) oak-solr-core test failures on Java 8
[ https://issues.apache.org/jira/browse/OAK-1819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-1819: - Fix Version/s: 1.2 oak-solr-core test failures on Java 8 - Key: OAK-1819 URL: https://issues.apache.org/jira/browse/OAK-1819 Project: Jackrabbit Oak Issue Type: Bug Components: oak-solr Affects Versions: 1.0 Environment: {noformat} Apache Maven 3.1.0 (893ca28a1da9d5f51ac03827af98bb730128f9f2; 2013-06-27 22:15:32-0400) Maven home: c:\Program Files\apache-maven-3.1.0 Java version: 1.8.0, vendor: Oracle Corporation Java home: c:\Program Files\Java\jdk1.8.0\jre Default locale: en_US, platform encoding: Cp1252 OS name: windows 7, version: 6.1, arch: amd64, family: dos {noformat} Reporter: Jukka Zitting Assignee: Tommaso Teofili Priority: Minor Labels: java8 Fix For: 1.2 The following {{oak-solr-core}} test failures occur when building Oak with Java 8: {noformat} Failed tests: testNativeMLTQuery(org.apache.jackrabbit.oak.plugins.index.solr.query.SolrIndexQueryTest): expected:/test/[a] but was:/test/[b] testNativeMLTQueryWithStream(org.apache.jackrabbit.oak.plugins.index.solr.query.SolrIndexQueryTest): expected:/test/[a] but was:/test/[b] {noformat} The cause of this might well be something as simple as the test case incorrectly expecting a specific ordering of search results. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2182) Specify collection to be used by Solr index
[ https://issues.apache.org/jira/browse/OAK-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2182: - Fix Version/s: (was: 1.1.8) Specify collection to be used by Solr index --- Key: OAK-2182 URL: https://issues.apache.org/jira/browse/OAK-2182 Project: Jackrabbit Oak Issue Type: Improvement Components: oak-solr Affects Versions: 1.1.0 Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.2 Currently all the information to hit a Solr server is hold by the singleton SolrServerProvider while there are some use cases where more than one query index definition for a Solr index may be done, targeting different content, and therefore it'd be good to be able to specify which collection should be used by each of these indexes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2510) Support for faceted search in Solr index
[ https://issues.apache.org/jira/browse/OAK-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2510: - Fix Version/s: (was: 1.2) 1.4 Support for faceted search in Solr index Key: OAK-2510 URL: https://issues.apache.org/jira/browse/OAK-2510 Project: Jackrabbit Oak Issue Type: Sub-task Components: oak-solr Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.4 A Solr index should be able to provide faceted search results, based on the support provided in the query engine. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2476) Move our CI to Jenkins
[ https://issues.apache.org/jira/browse/OAK-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2476: - Fix Version/s: (was: 1.1.8) Move our CI to Jenkins -- Key: OAK-2476 URL: https://issues.apache.org/jira/browse/OAK-2476 Project: Jackrabbit Oak Issue Type: Task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Priority: Critical Labels: CI, build, infrastructure Fix For: 1.2 We should strive for stabilization of our CI setup, as of now we had Buildbot and Travis. It seems ASF Jenkins can perform jobs on different environments (*nix, Windows and others) so we can evaluate that and check if it better address our needs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2611) Lucene suggester should only be updated if the index is used for suggestions
[ https://issues.apache.org/jira/browse/OAK-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2611: - Fix Version/s: (was: 1.1.8) 1.1.9 Lucene suggester should only be updated if the index is used for suggestions Key: OAK-2611 URL: https://issues.apache.org/jira/browse/OAK-2611 Project: Jackrabbit Oak Issue Type: Bug Components: oak-lucene Affects Versions: 1.1.7 Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.1.9 As mentioned on [oak-dev@|http://markmail.org/message/vp6qg7v5j3oxev73] Lucene suggester should be updated only in indexes that are used for suggestions (_useForSuggestions = true_ in index definition). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2477) Move suggester specific config to own configuration node
[ https://issues.apache.org/jira/browse/OAK-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2477: - Fix Version/s: (was: 1.1.8) Move suggester specific config to own configuration node Key: OAK-2477 URL: https://issues.apache.org/jira/browse/OAK-2477 Project: Jackrabbit Oak Issue Type: Improvement Components: oak-lucene Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.2 Currently suggester configuration is controlled via properties defined on main config / props node but it'd be good if we would have its own place to configure the whole suggest feature to not mix up configuration of other features / parameters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2264) Query engine expands like 'ab%d' to a very wide range
[ https://issues.apache.org/jira/browse/OAK-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-2264: --- Fix Version/s: (was: 1.2) 1.4 Query engine expands like 'ab%d' to a very wide range -- Key: OAK-2264 URL: https://issues.apache.org/jira/browse/OAK-2264 Project: Jackrabbit Oak Issue Type: Bug Components: query Affects Versions: 1.0.8, 1.1.2 Reporter: Amit Jain Fix For: 1.4 There is a problem with the range conversion for cases - like ab%d. Here the range obtained is (ab to ac) which would also return for e.g. abe, abae which is incorrect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2629) Cleanup Oak Travis jobs
[ https://issues.apache.org/jira/browse/OAK-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362938#comment-14362938 ] Marcel Reutegger commented on OAK-2629: --- Do we have MongoDB running on Jenkins for sufficient DocumentNodeStore on MongoDB test coverage? I would definitively like to keep at least one configuration on Travis for the MongoDB backend if Jenkins doesn't cover it. Alternatively we could also look into ways to start MongoDB with the help of a maven plugin. E.g. there's the [embedmongo-maven-plugin|https://github.com/joelittlejohn/embedmongo-maven-plugin]. I'm all for simplifying our CI usage, but only if this works on Jenkins. Cleanup Oak Travis jobs --- Key: OAK-2629 URL: https://issues.apache.org/jira/browse/OAK-2629 Project: Jackrabbit Oak Issue Type: Sub-task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.1.8, 1.2 Since we're moving toward Jenkins, let's remove the Travis jobs for Oak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1763) OrderedIndex does not comply with JCR's compareTo semantics
[ https://issues.apache.org/jira/browse/OAK-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-1763: --- Fix Version/s: (was: 1.2) 1.3.0 OrderedIndex does not comply with JCR's compareTo semantics --- Key: OAK-1763 URL: https://issues.apache.org/jira/browse/OAK-1763 Project: Jackrabbit Oak Issue Type: Bug Components: core Reporter: Michael Dürig Fix For: 1.3.0 The ordered index currently uses the lexicographical order of the string representation of the values. This does not comply with [JCR's compareTo sementics | http://www.day.com/specs/jcr/2.0/3_Repository_Model.html#3.6.5.1%20CompareTo%20Semantics] for e.g. double values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2264) Query engine expands like 'ab%d' to a very wide range
[ https://issues.apache.org/jira/browse/OAK-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-2264: --- Priority: Minor (was: Major) Query engine expands like 'ab%d' to a very wide range -- Key: OAK-2264 URL: https://issues.apache.org/jira/browse/OAK-2264 Project: Jackrabbit Oak Issue Type: Bug Components: query Affects Versions: 1.0.8, 1.1.2 Reporter: Amit Jain Priority: Minor Fix For: 1.4 There is a problem with the range conversion for cases - like ab%d. Here the range obtained is (ab to ac) which would also return for e.g. abe, abae which is incorrect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2016) Make blob gc max age configurable in SegmentNodeStoreService
[ https://issues.apache.org/jira/browse/OAK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-2016: --- Fix Version/s: 1.3.1 Make blob gc max age configurable in SegmentNodeStoreService Key: OAK-2016 URL: https://issues.apache.org/jira/browse/OAK-2016 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Amit Jain Assignee: Amit Jain Priority: Minor Fix For: 1.3.1 Attachments: OAK-2016.patch The blob gc max age setting is not configurable when using {{SegmentNodeStoreService}}. This can be made configurable and will be useful for testing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2081) [Ordered Index] Incorrect index plan selection
[ https://issues.apache.org/jira/browse/OAK-2081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-2081: --- Priority: Minor (was: Major) [Ordered Index] Incorrect index plan selection --- Key: OAK-2081 URL: https://issues.apache.org/jira/browse/OAK-2081 Project: Jackrabbit Oak Issue Type: Bug Components: query Reporter: Amit Jain Priority: Minor Attachments: OAK-2081.patch When an ordered index is defined on multiple properties the plan selection seems to be incorrect. There are a few potential problems : * For or conditions, the cost is doubled because of incorrect condition in {{OrderedContentMirrorStoreStrategy#count}}. The condition should be {{cost = max}} instead of {{count max}} * If multiple properties are specified in the index definition, only the first property is indexed and others ignored. IMO only the first should be used when selecting the plan in {{OrderedPropertyIndexLookup#getIndexNode()}}, otherwise it may happen that the property restriction that is not indexed is used for the index plan. * In {{QueryImpl#getBestSelectorExecutionPlan}} limit is used to damp the cost which does not look right. E.g. limit is set to 10 and 2 plans with cost 190 and 110 are candidates, the plan ultimately selected out of the 2 after damping with the limit will be arbitrary (the first to be processed) as the actual estimated cost is effectively ignored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2630) Cleanup Oak jobs on buildbot
[ https://issues.apache.org/jira/browse/OAK-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2630: - Fix Version/s: (was: 1.1.8) Cleanup Oak jobs on buildbot Key: OAK-2630 URL: https://issues.apache.org/jira/browse/OAK-2630 Project: Jackrabbit Oak Issue Type: Sub-task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.2 Since we're moving towards Jenkins, let's remove the buildbot jobs for Oak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2629) Cleanup Oak Travis jobs
[ https://issues.apache.org/jira/browse/OAK-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated OAK-2629: - Fix Version/s: (was: 1.1.8) Cleanup Oak Travis jobs --- Key: OAK-2629 URL: https://issues.apache.org/jira/browse/OAK-2629 Project: Jackrabbit Oak Issue Type: Sub-task Reporter: Tommaso Teofili Assignee: Tommaso Teofili Fix For: 1.2 Since we're moving toward Jenkins, let's remove the Travis jobs for Oak. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1375) Posting a json payload to the http-binding changes the root jcr:primaryType
[ https://issues.apache.org/jira/browse/OAK-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella updated OAK-1375: -- Fix Version/s: (was: 1.2) Posting a json payload to the http-binding changes the root jcr:primaryType --- Key: OAK-1375 URL: https://issues.apache.org/jira/browse/OAK-1375 Project: Jackrabbit Oak Issue Type: Bug Components: http Affects Versions: 0.16 Reporter: Davide Giannella Priority: Minor If you post with curl a json payload to oak (oak-run and httpbinding) the jcr:primaryType of root changes from rep:root to nt:unstructured. *Step for reproducing* * Fire oak with oak-run {{org.apache.jackrabbit.oak.run.Main}} and parameters: {{server /path/to/oak-repo}} * check the rep:root is fine {noformat} curl -u admin:admin -H Accept: application/json http://localhost:8080 {jcr:primaryType:rep:root,jcr:system:{},rep:security:{},oak:index:{}} {noformat} * post a json payload for creating {{/content}} {noformat} curl -u admin:admin -H Accept: application/json -H Content-Type: application/json -X POST -d {\jcr:primaryType\:\oak:unstructured\} http://localhost:8080/content {noformat} * check and see the / changing to nt:unstructured {noformat} curl -u admin:admin -H Accept: application/json http://localhost:8080 {jcr:primaryType:nt:unstructured,content:{},jcr:system:{},rep:security:{},oak:index:{}} {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)