[jira] [Commented] (OAK-4888) Warn or fail queries above a configurable cost value
[
https://issues.apache.org/jira/browse/OAK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589745#comment-15589745
]
Alexander Klimetschek commented on OAK-4888:
Awesome!
Some notes/questions:
* Will this catch queries with a suboptimal index that still needs a traversal
of the index (or more costly things besides an efficient index lookup), say
ordering by an unindexed property? That was the reason for my original
cost-based idea; it seems there is a {{bestCost}} readily available in
{{QueryImpl.getBestSelectorExecutionPlan}} that could be used in addition to
checking for no index at all ({{bestIndex == null}})
* I think if it logs "too many" warnings this is clearly a problem of the
application (for the current no-index check) – either ensure there is an index
or set the {{option(traversal ok)}} on these queries. I don't know why a
missing index can ever be considered ok :)
* +1 for an osgi configuration; if people enable that, they want to make it the
default in their application configuration, so their development team catches
issues early, which is much easier via an osgi configuration than making sure
the system property is set when starting the oak-based system.
> Warn or fail queries above a configurable cost value
>
>
> Key: OAK-4888
> URL: https://issues.apache.org/jira/browse/OAK-4888
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Alexander Klimetschek
>Assignee: Thomas Mueller
> Labels: docs-impacting
> Fix For: 1.5.13
>
>
> *Problem:* It's easy to run into performance problems with queries that are
> not backed by an index or miss the right one. Developers writing these
> queries typically do not have the real large production data, and thus don't
> see that a query would scale badly, and would not see any traversal warnings,
> as these only happen with a large number of results.
> *Proposal:* Oak's query engine already calculates a cost estimate to make a
> decision which index to use, or even if there is any at all. This cost
> estimate could be used to find out if a query is not supported by an index or
> with one that is not suitable enough (e.g. ordering by property that is not
> indexed)
> If a query is above a certain cost value, a big warning could be put out or
> even the query could be made to fail (maybe a per query option, that you
> might want to have to "fail" by default to ensure people are not overlooking
> the problem). The cost limit should be configurable, as it might depend on
> the hardware power.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589706#comment-15589706
]
Francesco Mari commented on OAK-4958:
-
At r1765706 I used the try-with-resources statement consistently in the standby
tests. This prevents client and servers from leaking resources if an exception
occurs. Moreover, I spotted and fixed a couple of NPEs in the close() methods
both in the standby client and server.
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
> Attachments: BrokenNetworkTest-logs.zip
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589223#comment-15589223
]
Francesco Mari commented on OAK-4958:
-
I simplified the setup scenario for cold standby tests at r1765680. I had to
temporary ignore testClientAndServerEmptyConfig in MBeanTest, as it
consistently fail with the current setup code - and it shouldn't. This will
require additional investigation.
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
> Attachments: BrokenNetworkTest-logs.zip
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari reassigned OAK-4958:
---
Assignee: Francesco Mari
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
> Attachments: BrokenNetworkTest-logs.zip
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4915) Interrupt online revision cleanup on documentmk
[
https://issues.apache.org/jira/browse/OAK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger resolved OAK-4915.
---
Resolution: Fixed
Fix Version/s: 1.5.13
Implemented in trunk: http://svn.apache.org/r1765676
> Interrupt online revision cleanup on documentmk
> ---
>
> Key: OAK-4915
> URL: https://issues.apache.org/jira/browse/OAK-4915
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: core, documentmk
>Reporter: Alex Parvulescu
>Assignee: Marcel Reutegger
> Labels: management
> Fix For: 1.6, 1.5.13
>
>
> Sub task of OAK-4835 for the {{document}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4581) Persistent local journal for more reliable event generation
[
https://issues.apache.org/jira/browse/OAK-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588993#comment-15588993
]
Stefan Egli edited comment on OAK-4581 at 10/19/16 3:24 PM:
h4. Prototype based on a SwitchingObserver
[pushed a
prototype|https://github.com/stefan-egli/jackrabbit-oak/commit/a521599e89b62ed8d40af5ae0a987cb4a9b546b7]
to my github fork which tries to scetch an approach that would use a
SwitchingObserer in front of either a BackgroundObserver/ChangeProcessor pair
or a PersistingChangeProcessor/PersistingEventQueue pair. The disadvantage of
this approach is that there are quite a few new classes in play. The advantage
is that it clearly distinguishes between normal (live) mode - by using the
exiting, unchanged BackgroundObserver/ChangeProcessor pair - and a new
persistent mode. Switching between these modes is delicate and is thus
explicitly handled with extra switching modes. This conceptually works fine -
added a test that illustrates the switch.
h4. Prototype based on persistence embedded in the ChangeProcessor
An alternative to the above would be to embedd the persistence directly in the
ChangeProcessor. The contentChanged method there would inspect the queue size
and if too large, not deliver events to the listener anymore, but instead go
via a persistent queue. The advantage is that it's more enclosed in the
ChangeProcessor. The disadvantage is that if a listener would block it could
not prevent the queue from growing (but maybe supporting an indefinitely
blocking listener is not required). I'll look into how such an approach could
be implemented next.
was (Author: egli):
h4. Prototype based on a SwitchingObserver
[pushed a
prototype|https://github.com/stefan-egli/jackrabbit-oak/commit/a521599e89b62ed8d40af5ae0a987cb4a9b546b7]
to my github fork which tries to scetch an approach that would use a
SwitchingObserer in front of either a BackgroundObserver/ChangeProcessor pair
or a PersistingBackgroundObserver/PersistingEventQueue pair. The disadvantage
of this approach is that there are quite a few new classes in play. The
advantage is that it clearly distinguishes between normal (live) mode - by
using the exiting, unchanged BackgroundObserver/ChangeProcessor pair - and a
new persistent mode. Switching between these modes is delicate and is thus
explicitly handled with extra switching modes. This conceptually works fine -
added a test that illustrates the switch.
h4. Prototype based on persistence embedded in the ChangeProcessor
An alternative to the above would be to embedd the persistence directly in the
ChangeProcessor. The contentChanged method there would inspect the queue size
and if too large, not deliver events to the listener anymore, but instead go
via a persistent queue. The advantage is that it's more enclosed in the
ChangeProcessor. The disadvantage is that if a listener would block it could
not prevent the queue from growing (but maybe supporting an indefinitely
blocking listener is not required). I'll look into how such an approach could
be implemented next.
> Persistent local journal for more reliable event generation
> ---
>
> Key: OAK-4581
> URL: https://issues.apache.org/jira/browse/OAK-4581
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4581.v0.patch
>
>
> As discussed in OAK-2683 "hitting the observation queue limit" has multiple
> drawbacks. Quite a bit of work is done to make diff generation faster.
> However there are still chances of event queue getting filled up.
> This issue is meant to implement a persistent event journal. Idea here being
> # NodeStore would push the diff into a persistent store via a synchronous
> observer
> # Observors which are meant to handle such events in async way (by virtue of
> being wrapped in BackgroundObserver) would instead pull the events from this
> persisted journal
> h3. A - What is persisted
> h4. 1 - Serialized Root States and CommitInfo
> In this approach we just persist the root states in serialized form.
> * DocumentNodeStore - This means storing the root revision vector
> * SegmentNodeStore - {color:red}Q1 - What does serialized form of
> SegmentNodeStore root state looks like{color} - Possible the RecordId of
> "root" state
> Note that with OAK-4528 DocumentNodeStore can rely on persisted remote
> journal to determine the affected paths. Which reduces the need for
> persisting complete diff locally.
> Event generation logic would then "deserialize" the persisted root states and
> then generate the diff as currently done via NodeState comparison
> h4. 2 - Serialized commit
[jira] [Commented] (OAK-4900) Enable persistent caches by default
[ https://issues.apache.org/jira/browse/OAK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589028#comment-15589028 ] Marcel Reutegger commented on OAK-4900: --- Updated component property descriptions in DocumentNodeStoreService: http://svn.apache.org/r1765661 > Enable persistent caches by default > --- > > Key: OAK-4900 > URL: https://issues.apache.org/jira/browse/OAK-4900 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Labels: observation > Fix For: 1.6, 1.5.13 > > > The diff persistent cache is important for efficient processing of external > changes and should be enabled by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4961) Default repository.home in DocumentNodeStoreService hides framework property
[
https://issues.apache.org/jira/browse/OAK-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger resolved OAK-4961.
---
Resolution: Fixed
Fix Version/s: 1.5.13
Fixed in trunk: http://svn.apache.org/r1765617
> Default repository.home in DocumentNodeStoreService hides framework property
>
>
> Key: OAK-4961
> URL: https://issues.apache.org/jira/browse/OAK-4961
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, documentmk
>Affects Versions: 1.5.6
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
> Fix For: 1.6, 1.5.13
>
>
> The default configuration value for {{repository.home}} hides the well known
> Apache Sling framework property with the same name. The default value was
> introduced with OAK-4200.
> A default value is useful, but should only be used when there is no framework
> or component context property set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4581) Persistent local journal for more reliable event generation
[
https://issues.apache.org/jira/browse/OAK-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588993#comment-15588993
]
Stefan Egli commented on OAK-4581:
--
h4. Prototype based on a SwitchingObserver
[pushed a
prototype|https://github.com/stefan-egli/jackrabbit-oak/commit/a521599e89b62ed8d40af5ae0a987cb4a9b546b7]
to my github fork which tries to scetch an approach that would use a
SwitchingObserer in front of either a BackgroundObserver/ChangeProcessor pair
or a PersistingBackgroundObserver/PersistingEventQueue pair. The disadvantage
of this approach is that there are quite a few new classes in play. The
advantage is that it clearly distinguishes between normal (live) mode - by
using the exiting, unchanged BackgroundObserver/ChangeProcessor pair - and a
new persistent mode. Switching between these modes is delicate and is thus
explicitly handled with extra switching modes. This conceptually works fine -
added a test that illustrates the switch.
h4. Prototype based on persistence embedded in the ChangeProcessor
An alternative to the above would be to embedd the persistence directly in the
ChangeProcessor. The contentChanged method there would inspect the queue size
and if too large, not deliver events to the listener anymore, but instead go
via a persistent queue. The advantage is that it's more enclosed in the
ChangeProcessor. The disadvantage is that if a listener would block it could
not prevent the queue from growing (but maybe supporting an indefinitely
blocking listener is not required). I'll look into how such an approach could
be implemented next.
> Persistent local journal for more reliable event generation
> ---
>
> Key: OAK-4581
> URL: https://issues.apache.org/jira/browse/OAK-4581
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4581.v0.patch
>
>
> As discussed in OAK-2683 "hitting the observation queue limit" has multiple
> drawbacks. Quite a bit of work is done to make diff generation faster.
> However there are still chances of event queue getting filled up.
> This issue is meant to implement a persistent event journal. Idea here being
> # NodeStore would push the diff into a persistent store via a synchronous
> observer
> # Observors which are meant to handle such events in async way (by virtue of
> being wrapped in BackgroundObserver) would instead pull the events from this
> persisted journal
> h3. A - What is persisted
> h4. 1 - Serialized Root States and CommitInfo
> In this approach we just persist the root states in serialized form.
> * DocumentNodeStore - This means storing the root revision vector
> * SegmentNodeStore - {color:red}Q1 - What does serialized form of
> SegmentNodeStore root state looks like{color} - Possible the RecordId of
> "root" state
> Note that with OAK-4528 DocumentNodeStore can rely on persisted remote
> journal to determine the affected paths. Which reduces the need for
> persisting complete diff locally.
> Event generation logic would then "deserialize" the persisted root states and
> then generate the diff as currently done via NodeState comparison
> h4. 2 - Serialized commit diff and CommitInfo
> In this approach we can save the diff in JSOP form. The diff only contains
> information about affected path. Similar to what is current being stored in
> DocumentNodeStore journal
> h4. CommitInfo
> The commit info would also need to be serialized. So it needs to be ensure
> whatever is stored there can be serialized or re calculated
> h3. B - How it is persisted
> h4. 1 - Use a secondary segment NodeStore
> OAK-4180 makes use of SegmentNodeStore as a secondary store for caching.
> [~mreutegg] suggested that for persisted local journal we can also utilize a
> SegmentNodeStore instance. Care needs to be taken for compaction. Either via
> generation approach or relying on online compaction
> h4. 2- Make use of write ahead log implementations
> [~ianeboston] suggested that we can make use of some write ahead log
> implementation like [1], [2] or [3]
> h3. C - How changes get pulled
> Some points to consider for event generation logic
> # Would need a way to keep pointers to journal entry on per listener basis.
> This would allow each Listener to "pull" content changes and generate diff as
> per its speed and keeping in memory overhead low
> # The journal should survive restarts
> [1] http://www.mapdb.org/javadoc/latest/mapdb/org/mapdb/WriteAheadLog.html
> [2]
> https://github.com/apache/activemq/tree/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal
> [3]
>
[jira] [Created] (OAK-4962) Standalone tooling for segment tar
Michael Dürig created OAK-4962:
--
Summary: Standalone tooling for segment tar
Key: OAK-4962
URL: https://issues.apache.org/jira/browse/OAK-4962
Project: Jackrabbit Oak
Issue Type: New Feature
Components: segment-tar
Reporter: Michael Dürig
Assignee: Michael Dürig
Fix For: Segment Tar 0.0.18
Currently there is no easy way to run the tools provided by {{oak-run}} against
a snapshot version of Segment Tar. In order to have better CI coverage (e.g.
benchmarks) of Segment Tar, we need to introduce a way for running such tools
independently of {{oak-run}}. Eventually {{oak-run}} should even be using that
tooling front-end instead of directly depending on {{oak-segment-tar}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588919#comment-15588919
]
Francesco Mari commented on OAK-4958:
-
The issue seems to be an IOException complaining about too many open files. I
will investigate further and check if we are leaking file descriptors
somewhere, be it because of non-closed files or dangling sockets.
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
> Attachments: BrokenNetworkTest-logs.zip
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4961) Default repository.home in DocumentNodeStoreService hides framework property
[
https://issues.apache.org/jira/browse/OAK-4961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger updated OAK-4961:
--
Affects Version/s: 1.5.6
> Default repository.home in DocumentNodeStoreService hides framework property
>
>
> Key: OAK-4961
> URL: https://issues.apache.org/jira/browse/OAK-4961
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, documentmk
>Affects Versions: 1.5.6
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
> Fix For: 1.6
>
>
> The default configuration value for {{repository.home}} hides the well known
> Apache Sling framework property with the same name. The default value was
> introduced with OAK-4200.
> A default value is useful, but should only be used when there is no framework
> or component context property set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4961) Default repository.home in DocumentNodeStoreService hides framework property
Marcel Reutegger created OAK-4961:
-
Summary: Default repository.home in DocumentNodeStoreService hides
framework property
Key: OAK-4961
URL: https://issues.apache.org/jira/browse/OAK-4961
Project: Jackrabbit Oak
Issue Type: Bug
Components: core, documentmk
Reporter: Marcel Reutegger
Assignee: Marcel Reutegger
Fix For: 1.6
The default configuration value for {{repository.home}} hides the well known
Apache Sling framework property with the same name. The default value was
introduced with OAK-4200.
A default value is useful, but should only be used when there is no framework
or component context property set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4958:
---
Attachment: BrokenNetworkTest-logs.zip
Logs from the failed test run
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
> Attachments: BrokenNetworkTest-logs.zip
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4283) Align GCMonitor API with implementation
[
https://issues.apache.org/jira/browse/OAK-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-4283.
Resolution: Fixed
Resolving as fixed. All what's left to do is tracked in the sub tasks.
> Align GCMonitor API with implementation
>
>
> Key: OAK-4283
> URL: https://issues.apache.org/jira/browse/OAK-4283
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Alex Parvulescu
> Labels: api-change, compaction, gc
> Fix For: Segment Tar 0.0.16
>
>
> The argument taken by {{GCMonitor.compacted}} related to parameters of the
> compaction map. The latter has gone with OAK-3348. We need to come up with a
> way to adjust this API accordingly. Also it might make sense to broaden the
> scope of {{GCMonitor}} from its initial intent (logging) to a more general
> one as this is how it is already used e.g. by the {{RefreshOnGC}}
> implementation and for OAK-4096.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4911) Can not read node Property when move code into a Thread (Java)
[
https://issues.apache.org/jira/browse/OAK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588775#comment-15588775
]
amin zamani commented on OAK-4911:
--
Hi Marcel,
thank you very much for investigating with me your time in this issue -> thanks
for your help!
I have attached attached the java Class "JCROakTest.java" to this ticket. There
are maybe only 3 Things you may modify at the beginning of the class:
private static Mongo mongo = new MongoClient("localhost");
private static String databaseName = "jcr-dev";
private String mixinDefs = "/mixin-definitions.cnd";
Inside the main start method (public static void main) you see that I create a
document with no thread - that works. And then afterward I create a document
inside a thread. The same code but only inside a thread.
But unfortunately now I have the problem that the document creation inside the
thread shows to me the error:
=> Exception in thread "Thread-1" java.lang.RuntimeException:
javax.jcr.RepositoryException: This session has been closed.
I use the same logic, exactly the same code inside our web application, the
same kind of session but I don't know why we get in our webapp this error:
Caused by: javax.jcr.PathNotFoundException: noah:createdBy not found on
/jcr:system/jcr:versionStorage/d2/88/43/d28843a2-c666-4936-b08b-e53912da4ecf/1.0/jcr:frozenNode
However, I think when we are able to fix the first problem (session closed)
then the second error will appear.
Thank you very much for your appreciated help!
Shalom
Amin
> Can not read node Property when move code into a Thread (Java)
> --
>
> Key: OAK-4911
> URL: https://issues.apache.org/jira/browse/OAK-4911
> Project: Jackrabbit Oak
> Issue Type: Bug
>Affects Versions: 1.3.10
> Environment: MAC OS X
>Reporter: amin zamani
> Attachments: java-code.txt
>
>
> Hallo,
> we develop an application and everything is working as it should. We upload
> documents and for each document a new preview file (node) should be
> generated. It works well when we execute the code synchron. But because of
> the fact that the generation of a preview file (like PDF format) can take
> very long time we decided to move the preview generation code into a usual
> java thread so that after the user is upload a document the website is
> responding very quick and does not wait till the preview file is generated
> but the preview file is generated parallel.
> No the problem: As soon as I move the same code regarding to the preview file
> generation into a usual thread (no modifications to the logic, only a thread
> is bordered to it) suddenly the node path does not exist, this is the error:
> -
> Caused by: javax.jcr.PathNotFoundException: jcr:lastModifiedBy not found on
> /jcr:system/jcr:versionStorage/7e/f8/5b/7ef85b25-4598-4476-82df-446eb3a08a90/1.0/jcr:frozenNode/jcr:content
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:631)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:625)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:207)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getProperty(NodeImpl.java:625)
> at
> com.westernacher.noah.util.content_repository.JcrUtil.getStringProperty(JcrUtil.java:35)
> ---
> In my point of view this is a bug. This is the code that works:
> Example:
> /// 1) User Upload document through browser in our web app..
> // 2) ... document uploaded and saved in OAK CR
> // 3) Now generate the preview:
> updateDocumentPreview(documentId);
>
> Now the same in a thread that does not work:
> => I have only moved the method "updateDocumentPreview(documentId);" in a
> Thread (Java 8):
> //Same code as before except the last line replaced by following lines:
> Runnable run = ()-> {updateDocumentPreview(documentId);}
> new Thread(run).start(); //Start the thread to generate a document preview
> --
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4911) Can not read node Property when move code into a Thread (Java)
[
https://issues.apache.org/jira/browse/OAK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amin zamani updated OAK-4911:
-
Attachment: mixin-definitions.cnd
I have also attached the mixin-defintions (file mixin-definitions.cnd) that
you may lay at your project classpath path like:
src/main/resources/mixin-definitions.cnd
> Can not read node Property when move code into a Thread (Java)
> --
>
> Key: OAK-4911
> URL: https://issues.apache.org/jira/browse/OAK-4911
> Project: Jackrabbit Oak
> Issue Type: Bug
>Affects Versions: 1.3.10
> Environment: MAC OS X
>Reporter: amin zamani
> Attachments: JCROakTest.java, java-code.txt, mixin-definitions.cnd
>
>
> Hallo,
> we develop an application and everything is working as it should. We upload
> documents and for each document a new preview file (node) should be
> generated. It works well when we execute the code synchron. But because of
> the fact that the generation of a preview file (like PDF format) can take
> very long time we decided to move the preview generation code into a usual
> java thread so that after the user is upload a document the website is
> responding very quick and does not wait till the preview file is generated
> but the preview file is generated parallel.
> No the problem: As soon as I move the same code regarding to the preview file
> generation into a usual thread (no modifications to the logic, only a thread
> is bordered to it) suddenly the node path does not exist, this is the error:
> -
> Caused by: javax.jcr.PathNotFoundException: jcr:lastModifiedBy not found on
> /jcr:system/jcr:versionStorage/7e/f8/5b/7ef85b25-4598-4476-82df-446eb3a08a90/1.0/jcr:frozenNode/jcr:content
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:631)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:625)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:207)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getProperty(NodeImpl.java:625)
> at
> com.westernacher.noah.util.content_repository.JcrUtil.getStringProperty(JcrUtil.java:35)
> ---
> In my point of view this is a bug. This is the code that works:
> Example:
> /// 1) User Upload document through browser in our web app..
> // 2) ... document uploaded and saved in OAK CR
> // 3) Now generate the preview:
> updateDocumentPreview(documentId);
>
> Now the same in a thread that does not work:
> => I have only moved the method "updateDocumentPreview(documentId);" in a
> Thread (Java 8):
> //Same code as before except the last line replaced by following lines:
> Runnable run = ()-> {updateDocumentPreview(documentId);}
> new Thread(run).start(); //Start the thread to generate a document preview
> --
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4911) Can not read node Property when move code into a Thread (Java)
[
https://issues.apache.org/jira/browse/OAK-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amin zamani updated OAK-4911:
-
Attachment: JCROakTest.java
> Can not read node Property when move code into a Thread (Java)
> --
>
> Key: OAK-4911
> URL: https://issues.apache.org/jira/browse/OAK-4911
> Project: Jackrabbit Oak
> Issue Type: Bug
>Affects Versions: 1.3.10
> Environment: MAC OS X
>Reporter: amin zamani
> Attachments: JCROakTest.java, java-code.txt
>
>
> Hallo,
> we develop an application and everything is working as it should. We upload
> documents and for each document a new preview file (node) should be
> generated. It works well when we execute the code synchron. But because of
> the fact that the generation of a preview file (like PDF format) can take
> very long time we decided to move the preview generation code into a usual
> java thread so that after the user is upload a document the website is
> responding very quick and does not wait till the preview file is generated
> but the preview file is generated parallel.
> No the problem: As soon as I move the same code regarding to the preview file
> generation into a usual thread (no modifications to the logic, only a thread
> is bordered to it) suddenly the node path does not exist, this is the error:
> -
> Caused by: javax.jcr.PathNotFoundException: jcr:lastModifiedBy not found on
> /jcr:system/jcr:versionStorage/7e/f8/5b/7ef85b25-4598-4476-82df-446eb3a08a90/1.0/jcr:frozenNode/jcr:content
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:631)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$11.perform(NodeImpl.java:625)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:207)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.getProperty(NodeImpl.java:625)
> at
> com.westernacher.noah.util.content_repository.JcrUtil.getStringProperty(JcrUtil.java:35)
> ---
> In my point of view this is a bug. This is the code that works:
> Example:
> /// 1) User Upload document through browser in our web app..
> // 2) ... document uploaded and saved in OAK CR
> // 3) Now generate the preview:
> updateDocumentPreview(documentId);
>
> Now the same in a thread that does not work:
> => I have only moved the method "updateDocumentPreview(documentId);" in a
> Thread (Java 8):
> //Same code as before except the last line replaced by following lines:
> Runnable run = ()-> {updateDocumentPreview(documentId);}
> new Thread(run).start(); //Start the thread to generate a document preview
> --
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4888) Warn or fail queries above a configurable cost value
[ https://issues.apache.org/jira/browse/OAK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-4888: - Labels: docs-impacting (was: ) > Warn or fail queries above a configurable cost value > > > Key: OAK-4888 > URL: https://issues.apache.org/jira/browse/OAK-4888 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Alexander Klimetschek >Assignee: Thomas Mueller > Labels: docs-impacting > Fix For: 1.5.13 > > > *Problem:* It's easy to run into performance problems with queries that are > not backed by an index or miss the right one. Developers writing these > queries typically do not have the real large production data, and thus don't > see that a query would scale badly, and would not see any traversal warnings, > as these only happen with a large number of results. > *Proposal:* Oak's query engine already calculates a cost estimate to make a > decision which index to use, or even if there is any at all. This cost > estimate could be used to find out if a query is not supported by an index or > with one that is not suitable enough (e.g. ordering by property that is not > indexed) > If a query is above a certain cost value, a big warning could be put out or > even the query could be made to fail (maybe a per query option, that you > might want to have to "fail" by default to ensure people are not overlooking > the problem). The cost limit should be configurable, as it might depend on > the hardware power. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4888) Warn or fail queries above a configurable cost value
[ https://issues.apache.org/jira/browse/OAK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588664#comment-15588664 ] Chetan Mehrotra commented on OAK-4888: -- bq. option(traversal ok) This is nice! [~tmueller] JMX based config approach is not persistent. Was thinking to introduce a new OSGi component which on startup obtains a reference to the JMX and sets it up based on OSGi config. This would allow a user to change such setting easily via std OSGi config instead of relying on system properties wdyt? > Warn or fail queries above a configurable cost value > > > Key: OAK-4888 > URL: https://issues.apache.org/jira/browse/OAK-4888 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Alexander Klimetschek >Assignee: Thomas Mueller > Fix For: 1.5.13 > > > *Problem:* It's easy to run into performance problems with queries that are > not backed by an index or miss the right one. Developers writing these > queries typically do not have the real large production data, and thus don't > see that a query would scale badly, and would not see any traversal warnings, > as these only happen with a large number of results. > *Proposal:* Oak's query engine already calculates a cost estimate to make a > decision which index to use, or even if there is any at all. This cost > estimate could be used to find out if a query is not supported by an index or > with one that is not suitable enough (e.g. ordering by property that is not > indexed) > If a query is above a certain cost value, a big warning could be put out or > even the query could be made to fail (maybe a per query option, that you > might want to have to "fail" by default to ensure people are not overlooking > the problem). The cost limit should be configurable, as it might depend on > the hardware power. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-937) Query engine index selection tweaks: shortcut and hint
[ https://issues.apache.org/jira/browse/OAK-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588652#comment-15588652 ] Thomas Mueller commented on OAK-937: A "option(...)" mechanism (similar to MS SQL Server) was introduced in OAK-4888. > Query engine index selection tweaks: shortcut and hint > -- > > Key: OAK-937 > URL: https://issues.apache.org/jira/browse/OAK-937 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Reporter: Alex Parvulescu >Assignee: Thomas Mueller >Priority: Minor > Labels: performance > Fix For: 1.6 > > > This issue covers 2 different changes related to the way the QueryEngine > selects a query index: > Firstly there could be a way to end the index selection process early via a > known constant value: if an index returns a known value token (like -1000) > then the query engine would effectively stop iterating through the existing > index impls and use that index directly. > Secondly it would be nice to be able to specify a desired index (if one is > known to perform better) thus skipping the existing selection mechanism (cost > calculation and comparison). This could be done via certain query hints [0]. > [0] http://en.wikipedia.org/wiki/Hint_(SQL) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-937) Query engine index selection tweaks: shortcut and hint
[ https://issues.apache.org/jira/browse/OAK-937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller reassigned OAK-937: -- Assignee: Thomas Mueller > Query engine index selection tweaks: shortcut and hint > -- > > Key: OAK-937 > URL: https://issues.apache.org/jira/browse/OAK-937 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Reporter: Alex Parvulescu >Assignee: Thomas Mueller >Priority: Minor > Labels: performance > Fix For: 1.6 > > > This issue covers 2 different changes related to the way the QueryEngine > selects a query index: > Firstly there could be a way to end the index selection process early via a > known constant value: if an index returns a known value token (like -1000) > then the query engine would effectively stop iterating through the existing > index impls and use that index directly. > Secondly it would be nice to be able to specify a desired index (if one is > known to perform better) thus skipping the existing selection mechanism (cost > calculation and comparison). This could be done via certain query hints [0]. > [0] http://en.wikipedia.org/wiki/Hint_(SQL) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4888) Warn or fail queries above a configurable cost value
[ https://issues.apache.org/jira/browse/OAK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller resolved OAK-4888. - Resolution: Fixed I consider this solved for now, however some "fine-tuning" might be needed. For example, it now might now log "too many" warnings (for some value of "too many"). I guess for that, it's best to create a new issue. > Warn or fail queries above a configurable cost value > > > Key: OAK-4888 > URL: https://issues.apache.org/jira/browse/OAK-4888 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Alexander Klimetschek >Assignee: Thomas Mueller > Fix For: 1.5.13 > > > *Problem:* It's easy to run into performance problems with queries that are > not backed by an index or miss the right one. Developers writing these > queries typically do not have the real large production data, and thus don't > see that a query would scale badly, and would not see any traversal warnings, > as these only happen with a large number of results. > *Proposal:* Oak's query engine already calculates a cost estimate to make a > decision which index to use, or even if there is any at all. This cost > estimate could be used to find out if a query is not supported by an index or > with one that is not suitable enough (e.g. ordering by property that is not > indexed) > If a query is above a certain cost value, a big warning could be put out or > even the query could be made to fail (maybe a per query option, that you > might want to have to "fail" by default to ensure people are not overlooking > the problem). The cost limit should be configurable, as it might depend on > the hardware power. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4888) Warn or fail queries above a configurable cost value
[
https://issues.apache.org/jira/browse/OAK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588631#comment-15588631
]
Thomas Mueller edited comment on OAK-4888 at 10/19/16 12:34 PM:
http://svn.apache.org/r1765583 (trunk)
By default, queries without index will log a warning as follows: {{Traversal
query (query without index): \{statement\}; consider creating an index}}.
Remark: no warning is logged if an index is available, but traversing is cheap
(because there are currently not that many nodes in the specified path).
There is a way to make those queries without index throw an exception instead
of just logging a warning, by setting the JMX QueryEngineSettings.failTraversal
to true.
In the query itself the syntax {{option(traversal \{ok|fail|warn\})}} is
supported (at the very end of the statement, after "order by"). This is to
override the default setting, for queries that traverse a well known number of
nodes (for example 10 or 20 nodes). This is supported for both XPath and SQL-2,
as follows:
{noformat}
/jcr:root/oak:index/*[@type='lucene'] option(traversal ok)
select * from [nt:base]
where ischildnode('/oak:index')
order by name()
option(traversal ok)
{noformat}
was (Author: tmueller):
http://svn.apache.org/r1765583 (trunk)
By default, queries without index will log a warning as follows: "Traversal
query (query without index): {statement}; consider creating an index". Remark:
no warning is logged if an index is available, but traversing is cheap (because
there are currently not that many nodes in the specified path).
There is a way to make those queries without index throw an exception instead
of just logging a warning, by setting the JMX QueryEngineSettings.failTraversal
to true.
In the query itself the syntax {{"option(traversal \{ok|fail|warn\})"}} is
supported (at the very end of the statement, after "order by"). This is to
override the default setting, for queries that traverse a well known number of
nodes (for example 10 or 20 nodes). This is supported for both XPath and SQL-2,
as follows:
{noformat}
/jcr:root/oak:index/*[@type='lucene'] option(traversal ok)
select * from [nt:base]
where ischildnode('/oak:index')
order by name()
option(traversal ok)
{noformat}
> Warn or fail queries above a configurable cost value
>
>
> Key: OAK-4888
> URL: https://issues.apache.org/jira/browse/OAK-4888
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Alexander Klimetschek
>Assignee: Thomas Mueller
> Fix For: 1.5.13
>
>
> *Problem:* It's easy to run into performance problems with queries that are
> not backed by an index or miss the right one. Developers writing these
> queries typically do not have the real large production data, and thus don't
> see that a query would scale badly, and would not see any traversal warnings,
> as these only happen with a large number of results.
> *Proposal:* Oak's query engine already calculates a cost estimate to make a
> decision which index to use, or even if there is any at all. This cost
> estimate could be used to find out if a query is not supported by an index or
> with one that is not suitable enough (e.g. ordering by property that is not
> indexed)
> If a query is above a certain cost value, a big warning could be put out or
> even the query could be made to fail (maybe a per query option, that you
> might want to have to "fail" by default to ensure people are not overlooking
> the problem). The cost limit should be configurable, as it might depend on
> the hardware power.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4888) Warn or fail queries above a configurable cost value
[ https://issues.apache.org/jira/browse/OAK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-4888: Fix Version/s: (was: 1.6) 1.5.13 > Warn or fail queries above a configurable cost value > > > Key: OAK-4888 > URL: https://issues.apache.org/jira/browse/OAK-4888 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Alexander Klimetschek >Assignee: Thomas Mueller > Fix For: 1.5.13 > > > *Problem:* It's easy to run into performance problems with queries that are > not backed by an index or miss the right one. Developers writing these > queries typically do not have the real large production data, and thus don't > see that a query would scale badly, and would not see any traversal warnings, > as these only happen with a large number of results. > *Proposal:* Oak's query engine already calculates a cost estimate to make a > decision which index to use, or even if there is any at all. This cost > estimate could be used to find out if a query is not supported by an index or > with one that is not suitable enough (e.g. ordering by property that is not > indexed) > If a query is above a certain cost value, a big warning could be put out or > even the query could be made to fail (maybe a per query option, that you > might want to have to "fail" by default to ensure people are not overlooking > the problem). The cost limit should be configurable, as it might depend on > the hardware power. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4888) Warn or fail queries above a configurable cost value
[
https://issues.apache.org/jira/browse/OAK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588631#comment-15588631
]
Thomas Mueller edited comment on OAK-4888 at 10/19/16 12:33 PM:
http://svn.apache.org/r1765583 (trunk)
By default, queries without index will log a warning as follows: "Traversal
query (query without index): {statement}; consider creating an index". Remark:
no warning is logged if an index is available, but traversing is cheap (because
there are currently not that many nodes in the specified path).
There is a way to make those queries without index throw an exception instead
of just logging a warning, by setting the JMX QueryEngineSettings.failTraversal
to true.
In the query itself the syntax {{"option(traversal \{ok|fail|warn\})"}} is
supported (at the very end of the statement, after "order by"). This is to
override the default setting, for queries that traverse a well known number of
nodes (for example 10 or 20 nodes). This is supported for both XPath and SQL-2,
as follows:
{noformat}
/jcr:root/oak:index/*[@type='lucene'] option(traversal ok)
select * from [nt:base]
where ischildnode('/oak:index')
order by name()
option(traversal ok)
{noformat}
was (Author: tmueller):
http://svn.apache.org/r1765583 (trunk)
By default, queries without index will log a warning as follows: "Traversal
query (query without index): {statement}; consider creating an index". Remark:
no warning is logged if an index is available, but traversing is cheap (because
there are currently not that many nodes in the specified path).
There is a way to make those queries without index throw an exception instead
of just logging a warning, by setting the JMX QueryEngineSettings.failTraversal
to true.
In the query itself the syntax "option(traversal {ok|fail|warn})" is supported
(at the very end of the statement, after "order by"). This is to override the
default setting, for queries that traverse a well known number of nodes (for
example 10 or 20 nodes). This is supported for both XPath and SQL-2, as follows:
{noformat}
/jcr:root/oak:index/*[@type='lucene'] option(traversal ok)
select * from [nt:base]
where ischildnode('/oak:index')
order by name()
option(traversal ok)
{noformat}
> Warn or fail queries above a configurable cost value
>
>
> Key: OAK-4888
> URL: https://issues.apache.org/jira/browse/OAK-4888
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Alexander Klimetschek
>Assignee: Thomas Mueller
> Fix For: 1.6
>
>
> *Problem:* It's easy to run into performance problems with queries that are
> not backed by an index or miss the right one. Developers writing these
> queries typically do not have the real large production data, and thus don't
> see that a query would scale badly, and would not see any traversal warnings,
> as these only happen with a large number of results.
> *Proposal:* Oak's query engine already calculates a cost estimate to make a
> decision which index to use, or even if there is any at all. This cost
> estimate could be used to find out if a query is not supported by an index or
> with one that is not suitable enough (e.g. ordering by property that is not
> indexed)
> If a query is above a certain cost value, a big warning could be put out or
> even the query could be made to fail (maybe a per query option, that you
> might want to have to "fail" by default to ensure people are not overlooking
> the problem). The cost limit should be configurable, as it might depend on
> the hardware power.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4888) Warn or fail queries above a configurable cost value
[
https://issues.apache.org/jira/browse/OAK-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588631#comment-15588631
]
Thomas Mueller commented on OAK-4888:
-
http://svn.apache.org/r1765583 (trunk)
By default, queries without index will log a warning as follows: "Traversal
query (query without index): {statement}; consider creating an index". Remark:
no warning is logged if an index is available, but traversing is cheap (because
there are currently not that many nodes in the specified path).
There is a way to make those queries without index throw an exception instead
of just logging a warning, by setting the JMX QueryEngineSettings.failTraversal
to true.
In the query itself the syntax "option(traversal {ok|fail|warn})" is supported
(at the very end of the statement, after "order by"). This is to override the
default setting, for queries that traverse a well known number of nodes (for
example 10 or 20 nodes). This is supported for both XPath and SQL-2, as follows:
{noformat}
/jcr:root/oak:index/*[@type='lucene'] option(traversal ok)
select * from [nt:base]
where ischildnode('/oak:index')
order by name()
option(traversal ok)
{noformat}
> Warn or fail queries above a configurable cost value
>
>
> Key: OAK-4888
> URL: https://issues.apache.org/jira/browse/OAK-4888
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Alexander Klimetschek
>Assignee: Thomas Mueller
> Fix For: 1.6
>
>
> *Problem:* It's easy to run into performance problems with queries that are
> not backed by an index or miss the right one. Developers writing these
> queries typically do not have the real large production data, and thus don't
> see that a query would scale badly, and would not see any traversal warnings,
> as these only happen with a large number of results.
> *Proposal:* Oak's query engine already calculates a cost estimate to make a
> decision which index to use, or even if there is any at all. This cost
> estimate could be used to find out if a query is not supported by an index or
> with one that is not suitable enough (e.g. ordering by property that is not
> indexed)
> If a query is above a certain cost value, a big warning could be put out or
> even the query could be made to fail (maybe a per query option, that you
> might want to have to "fail" by default to ensure people are not overlooking
> the problem). The cost limit should be configurable, as it might depend on
> the hardware power.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1312) Bundle nodes into a document
[ https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588549#comment-15588549 ] Julian Reschke commented on OAK-1312: - will do > Bundle nodes into a document > > > Key: OAK-1312 > URL: https://issues.apache.org/jira/browse/OAK-1312 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Chetan Mehrotra > Labels: performance > Fix For: 1.6 > > Attachments: OAK-1312-meta-prop-handling.patch, > OAK-1312-review-v1.diff, OAK-1312-review-v2.diff, benchmark-results.txt, > run-benchmark.sh > > > For very fine grained content with many nodes and only few properties per > node it would be more efficient to bundle multiple nodes into a single > MongoDB document. Mostly reading would benefit because there are less > roundtrips to the backend. At the same time storage footprint would be lower > because metadata overhead is per document. > Feature branch - > https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-1312 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4915) Interrupt online revision cleanup on documentmk
[
https://issues.apache.org/jira/browse/OAK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger updated OAK-4915:
--
Fix Version/s: 1.6
> Interrupt online revision cleanup on documentmk
> ---
>
> Key: OAK-4915
> URL: https://issues.apache.org/jira/browse/OAK-4915
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: core, documentmk
>Reporter: Alex Parvulescu
>Assignee: Marcel Reutegger
> Labels: management
> Fix For: 1.6
>
>
> Sub task of OAK-4835 for the {{document}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4915) Interrupt online revision cleanup on documentmk
[
https://issues.apache.org/jira/browse/OAK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger reassigned OAK-4915:
-
Assignee: Marcel Reutegger
> Interrupt online revision cleanup on documentmk
> ---
>
> Key: OAK-4915
> URL: https://issues.apache.org/jira/browse/OAK-4915
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: core, documentmk
>Reporter: Alex Parvulescu
>Assignee: Marcel Reutegger
> Labels: management
> Fix For: 1.6
>
>
> Sub task of OAK-4835 for the {{document}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4915) Interrupt online revision cleanup on documentmk
[
https://issues.apache.org/jira/browse/OAK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger updated OAK-4915:
--
Component/s: core
> Interrupt online revision cleanup on documentmk
> ---
>
> Key: OAK-4915
> URL: https://issues.apache.org/jira/browse/OAK-4915
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: core, documentmk
>Reporter: Alex Parvulescu
>Assignee: Marcel Reutegger
> Labels: management
> Fix For: 1.6
>
>
> Sub task of OAK-4835 for the {{document}} specific changes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-1312) Bundle nodes into a document
[
https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434914#comment-15434914
]
Julian Reschke edited comment on OAK-1312 at 10/19/16 11:50 AM:
Just a quick note: with the implementation, we can have potential holes in
documents - say we have {{/a, /a/b and /a/b/c}} and {{/a/b}} is bundled with
{{/a}}. Then we'd only have documents - {{0:/, 1:/a, 3:/a/b/c}}.
While that, of course, is ok, but we would need to factor this at least in
oak-mongo.js (I can't recall any place else right now) which loops recursively
at quite a few places with breaking the loop if no docs are found at a given
depth.
was (Author: catholicon):
Just a quick note: with the implementation, we can have potential holes in
documents - say we have {{/a, /a/b and /a/b/c}} and {{/a/b}} is bundled with
{{/a}}. Then we'd only have documents - {{0:/, 1:/a, 3:/a/b/c}}.
Whlie that, of course, is ok, but we would need to factor this at least in
oak-mongo.js (I can't recall any place else right now) which loops recursively
at quite a few places with breaking the loop if no docs are found at a given
depth.
> Bundle nodes into a document
>
>
> Key: OAK-1312
> URL: https://issues.apache.org/jira/browse/OAK-1312
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, documentmk
>Reporter: Marcel Reutegger
>Assignee: Chetan Mehrotra
> Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-1312-meta-prop-handling.patch,
> OAK-1312-review-v1.diff, OAK-1312-review-v2.diff, benchmark-results.txt,
> run-benchmark.sh
>
>
> For very fine grained content with many nodes and only few properties per
> node it would be more efficient to bundle multiple nodes into a single
> MongoDB document. Mostly reading would benefit because there are less
> roundtrips to the backend. At the same time storage footprint would be lower
> because metadata overhead is per document.
> Feature branch -
> https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-1312
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-1312) Bundle nodes into a document
[ https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15331177#comment-15331177 ] Julian Reschke edited comment on OAK-1312 at 10/19/16 11:44 AM: Linking to OAK-4471 as we start packing more stuff into Document we need to ensure that size can be controlled better was (Author: chetanm): Lining to OAK-4471 as we start packing more stuff into Document we need to ensure that size can be controlled better > Bundle nodes into a document > > > Key: OAK-1312 > URL: https://issues.apache.org/jira/browse/OAK-1312 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Chetan Mehrotra > Labels: performance > Fix For: 1.6 > > Attachments: OAK-1312-meta-prop-handling.patch, > OAK-1312-review-v1.diff, OAK-1312-review-v2.diff, benchmark-results.txt, > run-benchmark.sh > > > For very fine grained content with many nodes and only few properties per > node it would be more efficient to bundle multiple nodes into a single > MongoDB document. Mostly reading would benefit because there are less > roundtrips to the backend. At the same time storage footprint would be lower > because metadata overhead is per document. > Feature branch - > https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-1312 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-1312) Bundle nodes into a document
[
https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588423#comment-15588423
]
Chetan Mehrotra edited comment on OAK-1312 at 10/19/16 11:04 AM:
-
h4. Benchmark - Result with bundling enabled
Ran a benchmark using [script|^run-benchmark.sh] with
[results|^benchmark-results.txt]. Script also dumps Mongo DB stats, Metrics
stats etc. Results are also summarized
[here|https://docs.google.com/spreadsheets/d/1lzwDjwS-HSL0WazYBx9Wtx2ZI3J-fGl-EJ08-rxdAE8]
{noformat}
+--+---+-+-+-+--+--+-++-+-+++--+++--+
| Fixtues | C | min | 10% | 50% | 90% | max | N | Reader | Mutator |
Assets# | Mongo Doc# | Mongo Size | Idx Size | Find# | Query# | Comment
|
+--+---+-+-+-+--+--+-++-+-+++--+++--+
| Oak-Mongo-DS | 5 | 360 | 483 | 710 | 1509 | 2843 | 350 | 75251 | 2504|
3680| 56966 | 58 | 43 | 44387 | 2808 | #default
|
| Oak-Mongo-DS | 5 | 346 | 477 | 787 | 1508 | 2498 | 336 | 41805 | 1798|
3480| 8710 | 36 | 5| 5105 | 1906 | #bundling,ALL
|
| Oak-Mongo-DS | 5 | 312 | 469 | 746 | 1491 | 2630 | 339 | 67085 | 2268|
3550| 30162 | 58 | 22 | 26655 | 12008 |
#bundling,EXCLUDE_RENDITIONS |
+--+---+-+-+-+--+--+-++-+-+++--+++--+
{noformat}
*Environment details*
{noformat}
$ uname -a
Linux chetanm-laptop 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux
$ java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
$ mongo -version
MongoDB shell version: 2.6.4
{noformat}
*Legend*
* Mongo Doc# - number of Mongo documents across all collections
* Mongo Size - Size in MB of Mongo DB
* Idx Size - Size of all indexes in Mongo (MB)
* ALL - It uses bundling pattern {{jcr:content, jcr:content/metadata,
jcr:content/renditions/**}}
* EXCLUDE_RENDITIONS - It uses bundling pattern {{jcr:content,
jcr:content/metadata}}
*Highlights*
* With ALL bundling there is a significant reduction in
** Mongo docs - 56966 -> 8710
** Index size - 43 -> 5
** Calls to mongo for find
* BUT there is a decrease in read/write also
** Reads 75251 -> 41805
** Updates 2504 -> 1798
* Changing the bundling pattern helps in improving reads
So bundling leads to very signification savings in Mongo level storage. However
has some adverse impacts on read and updates.
*Next Steps*
* Merge current branch to trunk - As shown in previous comment if bundling is
disabled there is no perf imapct. So its safe in disabled state
* Analyze why reads have reduced - Given that access should involve lesser
number of remote calls we need to see why reads are slow
* benchmark in more real world scenarios where the read access pattern is more
real
* Benchmark on RDB - [~reschke] Can you run it against any DB setup you have
once I have done the merge to trunk
* Benchmark with Mongo 3.x - [~mreutegg] Can you try it against Wired Tiger
/cc [~mreutegg] [~catholicon] [~ianeboston] [~alexxx] [~mmarth] [~tmueller]
was (Author: chetanm):
h4. Benchmark - Result with bundling enabled
Ran a benchmark using [script|^run-benchmark.sh] with
[results|^benchmark-results.txt]. Script also dumps Mongo DB stats, Metrics
stats etc. Results are also summarized
[here|https://docs.google.com/spreadsheets/d/1lzwDjwS-HSL0WazYBx9Wtx2ZI3J-fGl-EJ08-rxdAE8]
{noformat}
+--+---+-+-+-+--+--+-++-+-+++--+++--+
| Fixtues | C | min | 10% | 50% | 90% | max | N | Reader | Mutator |
Assets# | Mongo Doc# | Mongo Size | Idx Size | Find# | Query# | Comment
|
+--+---+-+-+-+--+--+-++-+-+++--+++--+
| Oak-Mongo-DS | 5 | 360 | 483 | 710 | 1509 | 2843 | 350 | 75251 | 2504|
3680| 56966 | 58 | 43 | 44387 | 2808 | #default
|
| Oak-Mongo-DS | 5 | 346 | 477 | 787 | 1508 | 2498 | 336 | 41805 | 1798|
3480| 8710 | 36 | 5| 5105 | 1906 | #bundling,ALL
|
| Oak-Mongo-DS | 5 | 312 | 469 | 746 | 1491 | 2630 | 339 | 67085 | 2268|
3550| 30162 | 58 | 22 | 26655
[jira] [Commented] (OAK-1312) Bundle nodes into a document
[
https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588423#comment-15588423
]
Chetan Mehrotra commented on OAK-1312:
--
h4. Benchmark - Result with bundling enabled
Ran a benchmark using [script|^run-benchmark.sh] with
[results|^benchmark-results.txt]. Script also dumps Mongo DB stats, Metrics
stats etc. Results are also summarized
[here|https://docs.google.com/spreadsheets/d/1lzwDjwS-HSL0WazYBx9Wtx2ZI3J-fGl-EJ08-rxdAE8]
{noformat}
+--+---+-+-+-+--+--+-++-+-+++--+++--+
| Fixtues | C | min | 10% | 50% | 90% | max | N | Reader | Mutator |
Assets# | Mongo Doc# | Mongo Size | Idx Size | Find# | Query# | Comment
|
+--+---+-+-+-+--+--+-++-+-+++--+++--+
| Oak-Mongo-DS | 5 | 360 | 483 | 710 | 1509 | 2843 | 350 | 75251 | 2504|
3680| 56966 | 58 | 43 | 44387 | 2808 | #default
|
| Oak-Mongo-DS | 5 | 346 | 477 | 787 | 1508 | 2498 | 336 | 41805 | 1798|
3480| 8710 | 36 | 5| 5105 | 1906 | #bundling,ALL
|
| Oak-Mongo-DS | 5 | 312 | 469 | 746 | 1491 | 2630 | 339 | 67085 | 2268|
3550| 30162 | 58 | 22 | 26655 | 12008 |
#bundling,EXCLUDE_RENDITIONS |
+--+---+-+-+-+--+--+-++-+-+++--+++--+
{noformat}
*Environment details*
{noformat}
$ uname -a
Linux chetanm-laptop 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux
$ java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
$ mongo -version
MongoDB shell version: 2.6.4
{noformat}
*Legend*
* Mongo Doc# - number of Mongo documents across all collections
* Mongo Size - Size in MB of Mongo DB
* Idx Size - Size of all indexes in Mongo (MB)
* ALL - It uses bundling pattern {{jcr:content, jcr:content/metadata,
jcr:content/renditions/**}}
* EXCLUDE_RENDITIONS - It uses bundling pattern {{jcr:content,
jcr:content/metadata}}
*Highlights*
* With ALL bundling there is a significant reduction in
** Mongo docs - 56966 -> 8710
** Index size - 43 -> 5
** Calls to mongo for find
* BUT there is a decrease in read/write also
** Reads 75251 -> 41805
** Updates 2504 -> 1798
* Changing the bundling pattern helps in improving reads
So bundling leads to very signification savings in Mongo level storage. However
has some adverse impacts on read and updates.
*Next Steps*
* Merge current branch to trunk - As shown in previous comment if bundling is
disabled there is no perf imapct. So its safe in disabled state
* Analyze why reads have reduced - Given that access should involve lesser
number of remote calls we need to see why reads are slow
* benchmark in more real world scenarios where the read access pattern is more
real
* Benchmark on RDB - [~reschke] Can you run it against any DB setup you have
once I have done the merge to trunk
* Benchmark with Mongo 3.x - [~mreutegg] Can you try it against Wired Tiger
/cc [~mreutegg] [~catholicon] [~ianeboston] [~alexxx] [~mmarth]
> Bundle nodes into a document
>
>
> Key: OAK-1312
> URL: https://issues.apache.org/jira/browse/OAK-1312
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, documentmk
>Reporter: Marcel Reutegger
>Assignee: Chetan Mehrotra
> Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-1312-meta-prop-handling.patch,
> OAK-1312-review-v1.diff, OAK-1312-review-v2.diff, benchmark-results.txt,
> run-benchmark.sh
>
>
> For very fine grained content with many nodes and only few properties per
> node it would be more efficient to bundle multiple nodes into a single
> MongoDB document. Mostly reading would benefit because there are less
> roundtrips to the backend. At the same time storage footprint would be lower
> because metadata overhead is per document.
> Feature branch -
> https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-1312
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-1312) Bundle nodes into a document
[ https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-1312: - Attachment: benchmark-results.txt run-benchmark.sh > Bundle nodes into a document > > > Key: OAK-1312 > URL: https://issues.apache.org/jira/browse/OAK-1312 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Chetan Mehrotra > Labels: performance > Fix For: 1.6 > > Attachments: OAK-1312-meta-prop-handling.patch, > OAK-1312-review-v1.diff, OAK-1312-review-v2.diff, benchmark-results.txt, > run-benchmark.sh > > > For very fine grained content with many nodes and only few properties per > node it would be more efficient to bundle multiple nodes into a single > MongoDB document. Mostly reading would benefit because there are less > roundtrips to the backend. At the same time storage footprint would be lower > because metadata overhead is per document. > Feature branch - > https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-1312 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-1312) Bundle nodes into a document
[
https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585767#comment-15585767
]
Chetan Mehrotra edited comment on OAK-1312 at 10/19/16 10:20 AM:
-
h4. Benchmark
Implemented a benchmark and committed that to trunk in r1765455. It
has 3 parts
* main - The code here create an app:Asset structure
* Reader - This thread does reads of random assets and read the
'jcr:content/metadata/status' property
* Mutator - This thread modifies the status property of random asset nodes
{noformat}
java -jar oak-run*.jar benchmark --mongouri="mongodb://localhost:27017/oak"
BundlingNodeTest Oak-Mongo-DS
{noformat}
{{app:Asset}} looks like below
{noformat}
/{jcr:primaryType = oak:Asset}
jcr:content{jcr:primaryType = oak:Unstructured}
metadata{jcr:primaryType = oak:Unstructured, status = STARTING}
comments{jcr:primaryType = oak:Unstructured}
renditions{jcr:created = 2016-10-19T15:47:00.766+05:30, jcr:createdBy =
admin, jcr:primaryType = nt:folder}
thumbnail-0.png{jcr:created = 2016-10-19T15:47:00.771+05:30,
jcr:createdBy = admin, jcr:primaryType = nt:file}
jcr:content{jcr:lastModifiedBy = admin, jcr:mimeType = text/plain,
jcr:data = {5 bytes}, jcr:lastModified = 2016-10-19T15:47:00.781+05:30,
jcr:primaryType = oak:Resource}
thumbnail-1.png
...
thumbnail-2.png
...
thumbnail-3.png
...
thumbnail-4.png
{noformat}
h4. Benchmark - Overhead of new bundling logic when bundling *not* in use
It does not depend on bundling api and first purpose is to see overhead of new
bundling logic when bundling feature itself is not used.
{noformat}
# BundlingNodeTest C min 10% 50% 90% max
N Reader Mutator Assets#
trunk
Oak-Mongo-DS 1 189 280 432 8171414
123 7540328171390 #oak:Resource (trunk)
Oak-Mongo-DS 5 362 489 73516122740
340 7509324643580 #oak:Resource (trunk)
branch
Oak-Mongo-DS 1 224 292 453 7321591
122 7732827941400 #oak:Resource (OAK-1312 branch)
Oak-Mongo-DS 5 336 489 73514762787
349 7324524963660 #oak:Resource (OAK-1312 branch)
{noformat}
With above numbers it can be concluded that there is no overhead of bundling
related change in critical path
was (Author: chetanm):
h4. Benchmark
Implemented a benchmark and committed that to trunk in r1765455. It
has 3 parts
* main - The code here create an app:Asset structure
* Reader - This thread does reads of random assets and read the
'jcr:content/metadata/status' property
* Mutator - This thread modifies the status property of random asset nodes
{noformat}
java -jar oak-run*.jar benchmark --mongouri="mongodb://localhost:27017/oak"
BundlingNodeTest Oak-Mongo-DS
{noformat}
h4. Benchmark - Overhead of new bundling logic when bundling *not* in use
It does not depend on bundling api and first purpose is to see overhead of new
bundling logic when bundling feature itself is not used.
{noformat}
# BundlingNodeTest C min 10% 50% 90% max
N Reader Mutator Assets#
trunk
Oak-Mongo-DS 1 189 280 432 8171414
123 7540328171390 #oak:Resource (trunk)
Oak-Mongo-DS 5 362 489 73516122740
340 7509324643580 #oak:Resource (trunk)
branch
Oak-Mongo-DS 1 224 292 453 7321591
122 7732827941400 #oak:Resource (OAK-1312 branch)
Oak-Mongo-DS 5 336 489 73514762787
349 7324524963660 #oak:Resource (OAK-1312 branch)
{noformat}
With above numbers it can be concluded that there is no overhead of bundling
related change in critical path
> Bundle nodes into a document
>
>
> Key: OAK-1312
> URL: https://issues.apache.org/jira/browse/OAK-1312
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, documentmk
>Reporter: Marcel Reutegger
>Assignee: Chetan Mehrotra
> Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-1312-meta-prop-handling.patch,
> OAK-1312-review-v1.diff, OAK-1312-review-v2.diff
>
>
> For very fine grained content with many nodes and only few properties per
> node it would be more efficient to bundle multiple nodes into a single
> MongoDB document. Mostly reading would benefit because there are less
> roundtrips to the backend. At the same time storage footprint would be lower
> because metadata
[jira] [Commented] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588262#comment-15588262
]
Francesco Mari commented on OAK-4958:
-
The logs generated by that specific test should be in a subfolder of
target/failsafe-reports. I don't need the logs of every test.
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4960) Review the support for wildcards in bundling pattern
Chetan Mehrotra created OAK-4960: Summary: Review the support for wildcards in bundling pattern Key: OAK-4960 URL: https://issues.apache.org/jira/browse/OAK-4960 Project: Jackrabbit Oak Issue Type: Task Components: documentmk Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Fix For: 1.6 Bundling pattern currently supports wild card pattern. This makes it powerful but at same time can cause issue if it misconfigured. We should review this aspect before 1.6 release to determine if this feature needs to be exposed or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588250#comment-15588250
]
Michael Dürig commented on OAK-4958:
This is an AWS instance. I can share it with you.
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4959) Review the security aspect of bundling configuration
Chetan Mehrotra created OAK-4959:
Summary: Review the security aspect of bundling configuration
Key: OAK-4959
URL: https://issues.apache.org/jira/browse/OAK-4959
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Fix For: 1.6
The config for node bundling feature in DocumentNodeStore is currently stored
under {{jcr:system/rep:documentstore/bundlor}}. This task is meant to
* Review the access control aspect - This config should be only updatetable by
system admin
* Config under here should be writeable via JCR api
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588247#comment-15588247
]
Francesco Mari commented on OAK-4958:
-
If this happened in a Jenkins instance, can we collect the logs?
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4958) Test failure: BrokenNetworkTest
[
https://issues.apache.org/jira/browse/OAK-4958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4958:
---
Summary: Test failure: BrokenNetworkTest (was: Test failure: )
> Test failure: BrokenNetworkTest
> ---
>
> Key: OAK-4958
> URL: https://issues.apache.org/jira/browse/OAK-4958
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
>
> On some machines the {{BrokenNetworkTest}} fails:
> {noformat}
> Failed tests:
> BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{
> root = { ... } }> but was:<{ root : { } }>
> BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
>
> BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
> expected:<{ root = { ... } }> but was:<{ root : { } }>
> {noformat}
> Stack traces are all similar to
> {noformat}
> testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
> Time elapsed: 5.577 sec <<< FAILURE!
> java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { }
> }>
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
> at
> org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4958) Test failure:
Michael Dürig created OAK-4958:
--
Summary: Test failure:
Key: OAK-4958
URL: https://issues.apache.org/jira/browse/OAK-4958
Project: Jackrabbit Oak
Issue Type: Bug
Components: segment-tar
Reporter: Michael Dürig
Fix For: Segment Tar 0.0.18
On some machines the {{BrokenNetworkTest}} fails:
{noformat}
Failed tests:
BrokenNetworkTest.testProxyFlippedEndByteSSL:103->useProxy:146 expected:<{
root = { ... } }> but was:<{ root : { } }>
BrokenNetworkTest.testProxyFlippedIntermediateByte:88->useProxy:146
expected:<{ root = { ... } }> but was:<{ root : { } }>
BrokenNetworkTest.testProxyFlippedIntermediateByteSSL:93->useProxy:146
expected:<{ root = { ... } }> but was:<{ root : { } }>
BrokenNetworkTest.testProxyFlippedStartByte:78->useProxy:146 expected:<{ root
= { ... } }> but was:<{ root : { } }>
BrokenNetworkTest.testProxySSLSkippedBytes:63->useProxy:113->useProxy:146
expected:<{ root = { ... } }> but was:<{ root : { } }>
BrokenNetworkTest.testProxySSLSkippedBytesIntermediateChange:73->useProxy:113->useProxy:146
expected:<{ root = { ... } }> but was:<{ root : { } }>
BrokenNetworkTest.testProxySkippedBytesIntermediateChange:68->useProxy:113->useProxy:146
expected:<{ root = { ... } }> but was:<{ root : { } }>
{noformat}
Stack traces are all similar to
{noformat}
testProxySkippedBytesIntermediateChange(org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest)
Time elapsed: 5.577 sec <<< FAILURE!
java.lang.AssertionError: expected:<{ root = { ... } }> but was:<{ root : { } }>
at
org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:146)
at
org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.useProxy(BrokenNetworkTest.java:113)
at
org.apache.jackrabbit.oak.segment.standby.BrokenNetworkTest.testProxySkippedBytesIntermediateChange(BrokenNetworkTest.java:68)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-4617.
Resolution: Fixed
Resolving this issue. Further improvements are tracked with OAK-4957.
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588131#comment-15588131
]
Michael Dürig commented on OAK-4617:
Filed OAK-4957 for this.
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4957) SegmentRevisionGC MBean should report more detailed gc status information
Michael Dürig created OAK-4957: -- Summary: SegmentRevisionGC MBean should report more detailed gc status information Key: OAK-4957 URL: https://issues.apache.org/jira/browse/OAK-4957 Project: Jackrabbit Oak Issue Type: Improvement Components: segment-tar Reporter: Michael Dürig Fix For: Segment Tar 0.0.18 Regarding this, the current "Status" is showing the last log info. This is useful, but it would also be interesting to expose the real-time status. For monitoring it would be useful to know exactly in which phase we are, e.g. a field showing on of the following: - idle - estimation - compaction - compaction-retry-1 - compaction-retry-2 - compaction-forcecompact - cleanup -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4942) Upgrade Oak dependency to 1.5.13
[ https://issues.apache.org/jira/browse/OAK-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4942: --- Fix Version/s: (was: Segment Tar 0.0.16) Segment Tar 0.0.18 > Upgrade Oak dependency to 1.5.13 > > > Key: OAK-4942 > URL: https://issues.apache.org/jira/browse/OAK-4942 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig > Fix For: Segment Tar 0.0.18 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4945) Update GCMonitor api on segment-tar
[
https://issues.apache.org/jira/browse/OAK-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4945:
---
Fix Version/s: (was: Segment Tar 0.0.16)
Segment Tar 0.0.18
> Update GCMonitor api on segment-tar
> ---
>
> Key: OAK-4945
> URL: https://issues.apache.org/jira/browse/OAK-4945
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
> Fix For: Segment Tar 0.0.18
>
> Attachments: OAK-4945.patch
>
>
> {{segment-tar}} updates for OAK-4929
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4953) Optimise MutableRecordNumbers
[
https://issues.apache.org/jira/browse/OAK-4953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588081#comment-15588081
]
Michael Dürig commented on OAK-4953:
Benchmarks here:
https://issues.apache.org/jira/browse/OAK-4936?focusedCommentId=15588079=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15588079
> Optimise MutableRecordNumbers
> -
>
> Key: OAK-4953
> URL: https://issues.apache.org/jira/browse/OAK-4953
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: performance
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4953.patch
>
>
> {{MutableRecordNumbers}} is based on a map. This turns to be expensive as the
> items in the map are accessed very frequently. I would like to look into way
> of optimising this using a linear storage model instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4923) Improve segment deserialization performance
[
https://issues.apache.org/jira/browse/OAK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588080#comment-15588080
]
Michael Dürig commented on OAK-4923:
Benchmarks here:
https://issues.apache.org/jira/browse/OAK-4936?focusedCommentId=15588079=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15588079
> Improve segment deserialization performance
> ---
>
> Key: OAK-4923
> URL: https://issues.apache.org/jira/browse/OAK-4923
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Francesco Mari
>Assignee: Michael Dürig
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4923-01.patch, OAK-4923-02.patch
>
>
> The methods {{readReferencedSegments}} and {{readRecordNumberOffsets}} in
> {{Segment}} compute the returned data every time they are called. While this
> is a very clean implementation, this might force unexpected I/O operations,
> since the "buffer" the data is read from usually is a memory-mapped file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4936) Too many segment cache misses
[
https://issues.apache.org/jira/browse/OAK-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588079#comment-15588079
]
Michael Dürig commented on OAK-4936:
Some benchmarks showing the impact of this fix together with the fixes from
OAK-4953 and OAK-4923:
{noformat}
$ uname -a
Darwin mduerig-MacBookPro1 15.6.0 Darwin Kernel Version 15.6.0: Thu Jun 23
18:25:34 PDT 2016; root:xnu-3248.60.10~1/RELEASE_X86_64 x86_64
$ java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
$ java -Xmx2g -Druntime=300 -jar oak-run-1.6-SNAPSHOT.jar benchmark
ConcurrentReadWriteTest ConcurrentWriteTest ConcurrentCreateNodesTest Oak-Tar
Oak-Segment-Tar
Apache Jackrabbit Oak 1.6-SNAPSHOT
# ConcurrentReadWriteTest C min 10% 50% 90% max
N
Oak-Tar1 41 125 145 167 489
2047
Oak-Segment-Tar (base) 1 64 164 190 2142362
1477
Oak-Segment-Tar (fix) 1 59 145 171 195 887
1751
# ConcurrentWriteTest C min 10% 50% 90% max
N
Oak-Tar1 38 52 54 59 492
5461
Oak-Segment-Tar (base) 1 106 116 298 5721570
931
Oak-Segment-Tar (fix) 1 91 108 115 130 908
2506
# ConcurrentCreateNodesTestC min 10% 50% 90% max
N
Oak-Tar1 383 430 479 553 903
620
Oak-Segment-Tar (base) 1 699 731109913191873
275
Oak-Segment-Tar (fix) 1 699 737 89310361733
324
{noformat}
> Too many segment cache misses
> -
>
> Key: OAK-4936
> URL: https://issues.apache.org/jira/browse/OAK-4936
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: performance
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4936-monitoring.patch
>
>
> Running the {{ConcurrentWriteTest}} benchmark and monitoring the hits and
> misses of the segment cache (LIRS), I noticed that some segments are loaded
> over and over again (up to 3000 times).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4900) Enable persistent caches by default
[ https://issues.apache.org/jira/browse/OAK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15588075#comment-15588075 ] Marcel Reutegger commented on OAK-4900: --- Updated documentation: http://svn.apache.org/r1765557 > Enable persistent caches by default > --- > > Key: OAK-4900 > URL: https://issues.apache.org/jira/browse/OAK-4900 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Labels: observation > Fix For: 1.6, 1.5.13 > > > The diff persistent cache is important for efficient processing of external > changes and should be enabled by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4936) Too many segment cache misses
[
https://issues.apache.org/jira/browse/OAK-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-4936.
Resolution: Fixed
Fixed at http://svn.apache.org/viewvc?rev=1765554=rev by switching to the
Guava cache, which seems better suited here.
> Too many segment cache misses
> -
>
> Key: OAK-4936
> URL: https://issues.apache.org/jira/browse/OAK-4936
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: performance
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4936-monitoring.patch
>
>
> Running the {{ConcurrentWriteTest}} benchmark and monitoring the hits and
> misses of the segment cache (LIRS), I noticed that some segments are loaded
> over and over again (up to 3000 times).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4953) Optimise MutableRecordNumbers
[
https://issues.apache.org/jira/browse/OAK-4953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-4953.
Resolution: Fixed
Fixed at http://svn.apache.org/viewvc?rev=176=rev
> Optimise MutableRecordNumbers
> -
>
> Key: OAK-4953
> URL: https://issues.apache.org/jira/browse/OAK-4953
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: performance
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4953.patch
>
>
> {{MutableRecordNumbers}} is based on a map. This turns to be expensive as the
> items in the map are accessed very frequently. I would like to look into way
> of optimising this using a linear storage model instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4923) Improve segment deserialization performance
[
https://issues.apache.org/jira/browse/OAK-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-4923.
Resolution: Fixed
Fixed at http://svn.apache.org/viewvc?rev=1765553=rev
> Improve segment deserialization performance
> ---
>
> Key: OAK-4923
> URL: https://issues.apache.org/jira/browse/OAK-4923
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Francesco Mari
>Assignee: Michael Dürig
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4923-01.patch, OAK-4923-02.patch
>
>
> The methods {{readReferencedSegments}} and {{readRecordNumberOffsets}} in
> {{Segment}} compute the returned data every time they are called. While this
> is a very clean implementation, this might force unexpected I/O operations,
> since the "buffer" the data is read from usually is a memory-mapped file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4954) SetPropertyTest benchmark fails on Segment Tar
[
https://issues.apache.org/jira/browse/OAK-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-4954:
--
Assignee: Michael Dürig
> SetPropertyTest benchmark fails on Segment Tar
> --
>
> Key: OAK-4954
> URL: https://issues.apache.org/jira/browse/OAK-4954
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: test-failure
> Fix For: Segment Tar 0.0.18
>
>
> The {{SetPropertyTest}} fails on Oak Segment Tar:
> {noformat}
> javax.jcr.InvalidItemStateException: This item
> [/testfb3e8f1a/ca1ef350-f650-4466-b9e3-7f77d83e6303] does not exist anymore
> at
> org.apache.jackrabbit.oak.jcr.delegate.ItemDelegate.checkAlive(ItemDelegate.java:86)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl$ItemWriteOperation.checkPreconditions(ItemImpl.java:96)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl$35.checkPreconditions(NodeImpl.java:1366)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.prePerform(SessionDelegate.java:615)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.perform(SessionDelegate.java:205)
> at
> org.apache.jackrabbit.oak.jcr.session.ItemImpl.perform(ItemImpl.java:112)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.internalSetProperty(NodeImpl.java:1363)
> at
> org.apache.jackrabbit.oak.jcr.session.NodeImpl.setProperty(NodeImpl.java:506)
> at
> org.apache.jackrabbit.oak.benchmark.SetPropertyTest.runTest(SetPropertyTest.java:65)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.execute(AbstractTest.java:372)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.runTest(AbstractTest.java:221)
> at
> org.apache.jackrabbit.oak.benchmark.AbstractTest.run(AbstractTest.java:197)
> at
> org.apache.jackrabbit.oak.benchmark.BenchmarkRunner.main(BenchmarkRunner.java:456)
> at
> org.apache.jackrabbit.oak.run.BenchmarkCommand.execute(BenchmarkCommand.java:26)
> at org.apache.jackrabbit.oak.run.Mode.execute(Mode.java:63)
> at org.apache.jackrabbit.oak.run.Main.main(Main.java:49)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4742) Improve FileStoreStatsMBean
[ https://issues.apache.org/jira/browse/OAK-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4742: --- Fix Version/s: (was: Segment Tar 0.0.16) Segment Tar 0.0.18 > Improve FileStoreStatsMBean > --- > > Key: OAK-4742 > URL: https://issues.apache.org/jira/browse/OAK-4742 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig > Labels: monitoring > Fix For: Segment Tar 0.0.18 > > > We should add further data to that MBean (if feasible): > * Number of commits > * Number of commits queuing (blocked on the commit semaphore) > * Percentiles of commit times (exclude queueing time) > * Percentiles of commit queueing times > * Last gc run / size before gc and after gc / time gc took broken down into > the various phases -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4900) Enable persistent caches by default
[
https://issues.apache.org/jira/browse/OAK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger resolved OAK-4900.
---
Resolution: Fixed
Fix Version/s: 1.5.13
Done in trunk: http://svn.apache.org/r1765552
A persistent cache can be disabled when configured with a single dash
character. E.g. {{persistentCache=-}}. An empty or missing entry will use the
default and enable the persistent cache.
> Enable persistent caches by default
> ---
>
> Key: OAK-4900
> URL: https://issues.apache.org/jira/browse/OAK-4900
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, documentmk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Labels: observation
> Fix For: 1.6, 1.5.13
>
>
> The diff persistent cache is important for efficient processing of external
> changes and should be enabled by default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4900) Enable persistent caches by default
[ https://issues.apache.org/jira/browse/OAK-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-4900: -- Summary: Enable persistent caches by default (was: Enable diff persistent cache by default) Changed the summary because I also want to enable the regular persistent cache by default. For many setups this is beneficial because it helps reducing the number of calls to the DocumentStore backend. For now I will only change the default for the DocumentNodeStoreService. The DocumentMK.Builder is a slightly different story, because it defaults to an in-memory store, where a persistent cache doesn't make sense. > Enable persistent caches by default > --- > > Key: OAK-4900 > URL: https://issues.apache.org/jira/browse/OAK-4900 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Labels: observation > Fix For: 1.6 > > > The diff persistent cache is important for efficient processing of external > changes and should be enabled by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4955) Write log messages to file only (oak-pojosr)
[ https://issues.apache.org/jira/browse/OAK-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-4955. --- Resolution: Fixed Fix Version/s: 1.5.13 Done in trunk: http://svn.apache.org/r1765549 > Write log messages to file only (oak-pojosr) > > > Key: OAK-4955 > URL: https://issues.apache.org/jira/browse/OAK-4955 > Project: Jackrabbit Oak > Issue Type: Test > Components: pojosr >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.6, 1.5.13 > > > The log configuration for tests in oak-pojosr currently writes log messages > to a file and the console. This creates a rather verbose build output. I'd > like to disable the log output to the console. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4522) Improve CommitRateLimiter to optionally block some commits
[ https://issues.apache.org/jira/browse/OAK-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587982#comment-15587982 ] Thomas Mueller commented on OAK-4522: - http://svn.apache.org/r1765547 (fix test case, take 2) > Improve CommitRateLimiter to optionally block some commits > -- > > Key: OAK-4522 > URL: https://issues.apache.org/jira/browse/OAK-4522 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: jcr >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Labels: observation > Fix For: 1.5.12 > > Attachments: OAK-4522-b.patch, OAK-4522-c.patch, OAK-4522.patch > > > The CommitRateLimiter of OAK-1659 can delay commits, but doesn't currently > block them, and delays even those commits that are part of handling events. > Because of that, the queue can still get full, and possibly delaying commits > while handling events can make the situation even worse. > In Jackrabbit 2.x, we had a similar feature: JCR-2402. Also related is > JCR-2746. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587910#comment-15587910
]
Alex Parvulescu edited comment on OAK-4617 at 10/19/16 7:51 AM:
:) to clarify, I'd like to move this improvement discussion on a dedicated
issue.
was (Author: alex.parvulescu):
:)
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4916) Add support for excluding commits to BackgroundObserver
[
https://issues.apache.org/jira/browse/OAK-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587958#comment-15587958
]
Stefan Egli commented on OAK-4916:
--
[~chetanm], sure, no problem. I'll also look at the benchmark one of the next
days.
> Add support for excluding commits to BackgroundObserver
> ---
>
> Key: OAK-4916
> URL: https://issues.apache.org/jira/browse/OAK-4916
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: core
>Affects Versions: 1.5.11
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Fix For: 1.6
>
> Attachments: OAK-4916.patch, OAK-4916.v2.patch
>
>
> As part of pre-filtering commits it would be useful to have support in the
> BackgroundObserver (in general) that would allow to exclude certain commits
> from being added to the (BackgroundObserver's) queue, thus keeping the queue
> smaller. The actual filtering is up to subclasses.
> The suggested implementation is as follows:
> * a new method {{isExcluded}} is introduced which represents a subclass hook
> for filtering
> * excluded commits are not added to the queue
> * when multiple commits are excluded subsequently, this is collapsed
> * the first non-excluded commit (ContentChange) added to the queue is marked
> with the last non-excluded root state as the 'previous root'
> * downstream Observers are notified of the exclusion of a commit via a
> special CommitInfo {{NOOP_CHANGE}}: this instructs it to exclude this change
> while at the same time 'fast-forwarding' the root state to the new one.
> ** this extra token is one way of solving the problem that
> {{Observer.contentChanged}} represents a diff between two states but does not
> transport the 'from' state explicitly - that is implicitly taken from the
> previous call to {{contentChanged}}. Thus using such a gap token
> ({{NOOP_CHANGE}}) seems to be the only way to instruct Observers to skip a
> change.
> To repeat: whoever extends BackgroundObserver with filtering must be aware of
> the new {{NOOP_CHANGE}} token. Anyone not doing filtering will not get any
> {{NOOP_CHANGE}} tokens though.
> NOTE: See [comment further
> below|https://issues.apache.org/jira/browse/OAK-4916?focusedCommentId=15572165=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15572165]
> with a new suggested approach, which doesn't use NOOP_CHANGED but introduces
> a new FilteringAwareObserver instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4956) SegmentGCOptions retainedGenerations setting not exposed via OSGi
[
https://issues.apache.org/jira/browse/OAK-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4956:
---
Fix Version/s: Segment Tar 0.0.18
> SegmentGCOptions retainedGenerations setting not exposed via OSGi
> -
>
> Key: OAK-4956
> URL: https://issues.apache.org/jira/browse/OAK-4956
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Alex Parvulescu
> Fix For: Segment Tar 0.0.18
>
>
> Expose {{SegmentGCOptions#retainedGenerations}} setting via OSGi.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587930#comment-15587930
]
Alex Parvulescu commented on OAK-4617:
--
also found a TODO on {{SegmentRevisionGCMBean}} about this issue, we should
drop that as well when we close this.
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4956) SegmentGCOptions retainedGenerations setting not exposed via OSGi
Alex Parvulescu created OAK-4956:
Summary: SegmentGCOptions retainedGenerations setting not exposed
via OSGi
Key: OAK-4956
URL: https://issues.apache.org/jira/browse/OAK-4956
Project: Jackrabbit Oak
Issue Type: Bug
Components: segment-tar
Reporter: Alex Parvulescu
Expose {{SegmentGCOptions#retainedGenerations}} setting via OSGi.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587926#comment-15587926
]
Alex Parvulescu commented on OAK-4617:
--
OAK-4956
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4948) Prevent FileStore flush call before it's fully initialized
[
https://issues.apache.org/jira/browse/OAK-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu resolved OAK-4948.
--
Resolution: Fixed
Assignee: Alex Parvulescu
fixed with http://svn.apache.org/viewvc?rev=1765542=rev
> Prevent FileStore flush call before it's fully initialized
> --
>
> Key: OAK-4948
> URL: https://issues.apache.org/jira/browse/OAK-4948
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
> Fix For: Segment Tar 0.0.16
>
> Attachments: OAK-4948.patch
>
>
> As seen on a local test, it can sometimes happen that the {{flush}} call
> comes before the {{bind}} call.
> {noformat}
> java.lang.IllegalStateException: Revisions not bound to a store
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:150)
> ~[guava-15.0.jar:na]
> at
> org.apache.jackrabbit.oak.segment.file.TarRevisions.checkBound(TarRevisions.java:183)
> ~[classes/:na]
> at
> org.apache.jackrabbit.oak.segment.file.TarRevisions.flush(TarRevisions.java:198)
> ~[classes/:na]
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.flush(FileStore.java:382)
> ~[classes/:na]
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$3.run(FileStore.java:227)
> ~[classes/:na]
> at
> org.apache.jackrabbit.oak.segment.file.SafeRunnable.run(SafeRunnable.java:67)
> ~[classes/:na]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587910#comment-15587910
]
Alex Parvulescu commented on OAK-4617:
--
:)
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587909#comment-15587909
]
Michael Dürig commented on OAK-4617:
Good catch! This is an oversight and we should fix it. Could you report an
issue for this?
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587902#comment-15587902
]
Michael Dürig commented on OAK-4617:
No please don't ;-)
OAK-4941 is just the leftover technical task from OAK-4919 pending on the next
Oak release. I would suggest to keep discussing improvements here or on a
dedicated issue in itself.
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587895#comment-15587895
]
Alex Parvulescu commented on OAK-4617:
--
Just checked again. The {{SegmentGCOptions#retainedGenerations}} is not exposed
as OSGi, it can be updated via JMX though.
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587893#comment-15587893
]
Alex Parvulescu commented on OAK-4617:
--
[~volteanu] please add this to the dedicated issue OAK-4941
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4955) Write log messages to file only (oak-pojosr)
Marcel Reutegger created OAK-4955: - Summary: Write log messages to file only (oak-pojosr) Key: OAK-4955 URL: https://issues.apache.org/jira/browse/OAK-4955 Project: Jackrabbit Oak Issue Type: Test Components: pojosr Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Fix For: 1.6 The log configuration for tests in oak-pojosr currently writes log messages to a file and the console. This creates a rather verbose build output. I'd like to disable the log output to the console. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
