[jira] [Commented] (OAK-4821) Allow use of Java 7 in Oak 1.4
[ https://issues.apache.org/jira/browse/OAK-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509020#comment-15509020 ] Julian Reschke commented on OAK-4821: - (discussed on mailing list, seems to have consensus) > Allow use of Java 7 in Oak 1.4 > -- > > Key: OAK-4821 > URL: https://issues.apache.org/jira/browse/OAK-4821 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Affects Versions: 1.4.7 >Reporter: Julian Reschke >Assignee: Julian Reschke > > (mailing list discussion in progress) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Labels: candidate_oak_1_0 candidate_oak_1_2 (was: candidate_oak_1_0 candidate_oak_1_2 candidate_oak_1_4) > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2 > Fix For: 1.4.8, 1.5.11 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506491#comment-15506491 ] Julian Reschke edited comment on OAK-4583 at 9/21/16 7:51 AM: -- trunk: [r1761571|http://svn.apache.org/r1761571] 1.4: [r1761691|http://svn.apache.org/r1761691] was (Author: reschke): trunk: [r1761571|http://svn.apache.org/r1761571] > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2 > Fix For: 1.4.8, 1.5.11 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Fix Version/s: 1.4.8 > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2 > Fix For: 1.4.8, 1.5.11 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4831) Don't break the upgrade tests if the directory can't be cleaned-up
Tomek Rękawek created OAK-4831:
--
Summary: Don't break the upgrade tests if the directory can't be
cleaned-up
Key: OAK-4831
URL: https://issues.apache.org/jira/browse/OAK-4831
Project: Jackrabbit Oak
Issue Type: Improvement
Components: upgrade
Reporter: Tomek Rękawek
Fix For: 1.6
The oak-upgrade tests creates a lot of repositories. If a repository doesn't
release the directory it uses, the whole test fails. We should handle it more
gracefully:
a) log the exact reason why the directory can't be removed,
b) carry on with the tests,
c) create directories in the Maven target directory, so they can be easily
removed with {{mvn clean}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4831) Don't break the upgrade tests if the directory can't be cleaned-up
[
https://issues.apache.org/jira/browse/OAK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek reassigned OAK-4831:
--
Assignee: Tomek Rękawek
> Don't break the upgrade tests if the directory can't be cleaned-up
> --
>
> Key: OAK-4831
> URL: https://issues.apache.org/jira/browse/OAK-4831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
>
> The oak-upgrade tests creates a lot of repositories. If a repository doesn't
> release the directory it uses, the whole test fails. We should handle it more
> gracefully:
> a) log the exact reason why the directory can't be removed,
> b) carry on with the tests,
> c) create directories in the Maven target directory, so they can be easily
> removed with {{mvn clean}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4831) Don't break the upgrade tests if the directory can't be cleaned-up
[
https://issues.apache.org/jira/browse/OAK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4831:
---
Description:
The oak-upgrade tests creates a lot of repositories. If a repository doesn't
release the directory it uses, [the whole test
fails|http://jackrabbit.markmail.org/thread/lmffrxvki7nxm3qy]. We should handle
it more gracefully:
a) log the exact reason why the directory can't be removed,
b) carry on with the tests,
c) create directories in the Maven target directory, so they can be easily
removed with {{mvn clean}}
was:
The oak-upgrade tests creates a lot of repositories. If a repository doesn't
release the directory it uses, the whole test fails. We should handle it more
gracefully:
a) log the exact reason why the directory can't be removed,
b) carry on with the tests,
c) create directories in the Maven target directory, so they can be easily
removed with {{mvn clean}}
> Don't break the upgrade tests if the directory can't be cleaned-up
> --
>
> Key: OAK-4831
> URL: https://issues.apache.org/jira/browse/OAK-4831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
>
> The oak-upgrade tests creates a lot of repositories. If a repository doesn't
> release the directory it uses, [the whole test
> fails|http://jackrabbit.markmail.org/thread/lmffrxvki7nxm3qy]. We should
> handle it more gracefully:
> a) log the exact reason why the directory can't be removed,
> b) carry on with the tests,
> c) create directories in the Maven target directory, so they can be easily
> removed with {{mvn clean}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509153#comment-15509153
]
Stefan Egli commented on OAK-4796:
--
thx for the reviews! I'd look at fixing those points.
Before that would be good if we could decide which approach to take: the one
suggested via the patch or the one suggested by Chetan.
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> # events (in the form of ContentChange Objects) occupy a slot of the queue
> even if the listener is not interested in it - any commit lands on any
> listener's queue. This reduces the capacity of the queue for 'actual' events
> to be delivered. It therefore increases the risk that the queue fills - and
> when full has various consequences such as loosing the CommitInfo etc.
> # each event==ContentChange later on must be evaluated, and for that a diff
> must be calculated. Depending on runtime behavior that diff might be
> expensive if no longer in the cache (documentMk specifically).
> As an improvement, this diffing+filtering could be done at an earlier stage
> already, nearer to the commit, and in case the filter would ignore the event,
> it would not have to be put into the queue at all, thus avoiding occupying a
> slot and later potentially slower diffing.
> The suggestion is to implement this via the following algorithm:
> * During the commit, in a {{Validator}} the listener's filters are evaluated
> - in an as-efficient-as-possible manner (Reason for doing it in a Validator
> is that this doesn't add overhead as oak already goes through all changes for
> other Validators). As a result a _list of potentially affected observers_ is
> added to the {{CommitInfo}} (false positives are fine).
> ** Note that the above adds cost to the commit and must therefore be
> carefully done and measured
> ** One potential measure could be to only do filtering when listener's queues
> are larger than a certain threshold (eg 10)
> * The ChangeProcessor in {{contentChanged}} (in the one created in
> [createObserver|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L224])
> then checks the new commitInfo's _potentially affected observers_ list and
> if it's not in the list, adds a {{NOOP}} token at the end of the queue. If
> there's already a NOOP there, the two are collapsed (this way when a filter
> is not affected it would have a NOOP at the end of the queue). If later on a
> no-NOOP item is added, the NOOP's {{root}} is used as the {{previousRoot}}
> for the newly added {{ContentChange}} obj.
> ** To achieve that, the ContentChange obj is extended to not only have the
> "to" {{root}} pointer, but also the "from" {{previousRoot}} pointer which
> currently is implicitly maintained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4831) Don't break the upgrade tests if the directory can't be cleaned-up
[
https://issues.apache.org/jira/browse/OAK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek resolved OAK-4831.
Resolution: Fixed
> Don't break the upgrade tests if the directory can't be cleaned-up
> --
>
> Key: OAK-4831
> URL: https://issues.apache.org/jira/browse/OAK-4831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
>
> The oak-upgrade tests creates a lot of repositories. If a repository doesn't
> release the directory it uses, [the whole test
> fails|http://jackrabbit.markmail.org/thread/lmffrxvki7nxm3qy]. We should
> handle it more gracefully:
> a) log the exact reason why the directory can't be removed,
> b) carry on with the tests,
> c) create directories in the Maven target directory, so they can be easily
> removed with {{mvn clean}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4831) Don't break the upgrade tests if the directory can't be cleaned-up
[
https://issues.apache.org/jira/browse/OAK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509189#comment-15509189
]
Tomek Rękawek commented on OAK-4831:
Fixed for trunk in [r1761695|https://svn.apache.org/r1761695].
> Don't break the upgrade tests if the directory can't be cleaned-up
> --
>
> Key: OAK-4831
> URL: https://issues.apache.org/jira/browse/OAK-4831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
>
> The oak-upgrade tests creates a lot of repositories. If a repository doesn't
> release the directory it uses, [the whole test
> fails|http://jackrabbit.markmail.org/thread/lmffrxvki7nxm3qy]. We should
> handle it more gracefully:
> a) log the exact reason why the directory can't be removed,
> b) carry on with the tests,
> c) create directories in the Maven target directory, so they can be easily
> removed with {{mvn clean}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4831) Don't break the upgrade tests if the directory can't be cleaned-up
[
https://issues.apache.org/jira/browse/OAK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4831:
---
Fix Version/s: 1.5.11
> Don't break the upgrade tests if the directory can't be cleaned-up
> --
>
> Key: OAK-4831
> URL: https://issues.apache.org/jira/browse/OAK-4831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
>
> The oak-upgrade tests creates a lot of repositories. If a repository doesn't
> release the directory it uses, [the whole test
> fails|http://jackrabbit.markmail.org/thread/lmffrxvki7nxm3qy]. We should
> handle it more gracefully:
> a) log the exact reason why the directory can't be removed,
> b) carry on with the tests,
> c) create directories in the Maven target directory, so they can be easily
> removed with {{mvn clean}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4831) Don't break the upgrade tests if the directory can't be cleaned-up
[
https://issues.apache.org/jira/browse/OAK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509218#comment-15509218
]
Michael Dürig commented on OAK-4831:
bq. create directories in the Maven target directory
FTR, this is also important for the Apache Jenkins CI where /tmp is located on
a partition with limited space. See OAK-4416
> Don't break the upgrade tests if the directory can't be cleaned-up
> --
>
> Key: OAK-4831
> URL: https://issues.apache.org/jira/browse/OAK-4831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
>
> The oak-upgrade tests creates a lot of repositories. If a repository doesn't
> release the directory it uses, [the whole test
> fails|http://jackrabbit.markmail.org/thread/lmffrxvki7nxm3qy]. We should
> handle it more gracefully:
> a) log the exact reason why the directory can't be removed,
> b) carry on with the tests,
> c) create directories in the Maven target directory, so they can be easily
> removed with {{mvn clean}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4830) StringUtils.estimateMemoryUsage() can throw NullPointerException
[
https://issues.apache.org/jira/browse/OAK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger reassigned OAK-4830:
-
Assignee: Marcel Reutegger
> StringUtils.estimateMemoryUsage() can throw NullPointerException
>
>
> Key: OAK-4830
> URL: https://issues.apache.org/jira/browse/OAK-4830
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: commons
>Affects Versions: 1.5.10
>Reporter: Matt Ryan
>Assignee: Marcel Reutegger
> Attachments: OAK-4830.1.patch
>
>
> The {{estimateMemoryUsage()}} method in {{StringUtils}} in {{oak-commons}}
> doesn't check if the string being passed in is null, so there's potential to
> throw a {{NullPointerException}} if called with a null string.
> Suggest it returns 0 in this case. I have a patch ready.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4830) StringUtils.estimateMemoryUsage() can throw NullPointerException
[
https://issues.apache.org/jira/browse/OAK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger updated OAK-4830:
--
Priority: Minor (was: Major)
> StringUtils.estimateMemoryUsage() can throw NullPointerException
>
>
> Key: OAK-4830
> URL: https://issues.apache.org/jira/browse/OAK-4830
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: commons
>Affects Versions: 1.5.10
>Reporter: Matt Ryan
>Assignee: Marcel Reutegger
>Priority: Minor
> Fix For: 1.5.11
>
> Attachments: OAK-4830.1.patch
>
>
> The {{estimateMemoryUsage()}} method in {{StringUtils}} in {{oak-commons}}
> doesn't check if the string being passed in is null, so there's potential to
> throw a {{NullPointerException}} if called with a null string.
> Suggest it returns 0 in this case. I have a patch ready.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4830) StringUtils.estimateMemoryUsage() can throw NullPointerException
[
https://issues.apache.org/jira/browse/OAK-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger resolved OAK-4830.
---
Resolution: Fixed
Fix Version/s: 1.5.11
Thanks for the patch. I applied it to trunk: http://svn.apache.org/r1761696
> StringUtils.estimateMemoryUsage() can throw NullPointerException
>
>
> Key: OAK-4830
> URL: https://issues.apache.org/jira/browse/OAK-4830
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: commons
>Affects Versions: 1.5.10
>Reporter: Matt Ryan
>Assignee: Marcel Reutegger
> Fix For: 1.5.11
>
> Attachments: OAK-4830.1.patch
>
>
> The {{estimateMemoryUsage()}} method in {{StringUtils}} in {{oak-commons}}
> doesn't check if the string being passed in is null, so there's potential to
> throw a {{NullPointerException}} if called with a null string.
> Suggest it returns 0 in this case. I have a patch ready.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4831) Don't break the upgrade tests if the directory can't be cleaned-up
[
https://issues.apache.org/jira/browse/OAK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509290#comment-15509290
]
Tomek Rękawek commented on OAK-4831:
Thanks, I wasn't aware about this. In the
[r1761700|https://svn.apache.org/r1761700] I fixed all the remaining tests in
the oak-upgrade to use the ./target as the temporary directory.
> Don't break the upgrade tests if the directory can't be cleaned-up
> --
>
> Key: OAK-4831
> URL: https://issues.apache.org/jira/browse/OAK-4831
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
>
> The oak-upgrade tests creates a lot of repositories. If a repository doesn't
> release the directory it uses, [the whole test
> fails|http://jackrabbit.markmail.org/thread/lmffrxvki7nxm3qy]. We should
> handle it more gracefully:
> a) log the exact reason why the directory can't be removed,
> b) carry on with the tests,
> c) create directories in the Maven target directory, so they can be easily
> removed with {{mvn clean}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4581) Persistent local journal for more reliable event generation
[
https://issues.apache.org/jira/browse/OAK-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509287#comment-15509287
]
Stefan Egli commented on OAK-4581:
--
/cc [~mreutegg]
> Persistent local journal for more reliable event generation
> ---
>
> Key: OAK-4581
> URL: https://issues.apache.org/jira/browse/OAK-4581
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4581.v0.patch
>
>
> As discussed in OAK-2683 "hitting the observation queue limit" has multiple
> drawbacks. Quite a bit of work is done to make diff generation faster.
> However there are still chances of event queue getting filled up.
> This issue is meant to implement a persistent event journal. Idea here being
> # NodeStore would push the diff into a persistent store via a synchronous
> observer
> # Observors which are meant to handle such events in async way (by virtue of
> being wrapped in BackgroundObserver) would instead pull the events from this
> persisted journal
> h3. A - What is persisted
> h4. 1 - Serialized Root States and CommitInfo
> In this approach we just persist the root states in serialized form.
> * DocumentNodeStore - This means storing the root revision vector
> * SegmentNodeStore - {color:red}Q1 - What does serialized form of
> SegmentNodeStore root state looks like{color} - Possible the RecordId of
> "root" state
> Note that with OAK-4528 DocumentNodeStore can rely on persisted remote
> journal to determine the affected paths. Which reduces the need for
> persisting complete diff locally.
> Event generation logic would then "deserialize" the persisted root states and
> then generate the diff as currently done via NodeState comparison
> h4. 2 - Serialized commit diff and CommitInfo
> In this approach we can save the diff in JSOP form. The diff only contains
> information about affected path. Similar to what is current being stored in
> DocumentNodeStore journal
> h4. CommitInfo
> The commit info would also need to be serialized. So it needs to be ensure
> whatever is stored there can be serialized or re calculated
> h3. B - How it is persisted
> h4. 1 - Use a secondary segment NodeStore
> OAK-4180 makes use of SegmentNodeStore as a secondary store for caching.
> [~mreutegg] suggested that for persisted local journal we can also utilize a
> SegmentNodeStore instance. Care needs to be taken for compaction. Either via
> generation approach or relying on online compaction
> h4. 2- Make use of write ahead log implementations
> [~ianeboston] suggested that we can make use of some write ahead log
> implementation like [1], [2] or [3]
> h3. C - How changes get pulled
> Some points to consider for event generation logic
> # Would need a way to keep pointers to journal entry on per listener basis.
> This would allow each Listener to "pull" content changes and generate diff as
> per its speed and keeping in memory overhead low
> # The journal should survive restarts
> [1] http://www.mapdb.org/javadoc/latest/mapdb/org/mapdb/WriteAheadLog.html
> [2]
> https://github.com/apache/activemq/tree/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal
> [3]
> https://github.com/elastic/elasticsearch/tree/master/core/src/main/java/org/elasticsearch/index/translog
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4832) Upgrade breaks if the SecurityManager section in repository.xml is empty
[
https://issues.apache.org/jira/browse/OAK-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek reassigned OAK-4832:
--
Assignee: Tomek Rękawek
> Upgrade breaks if the SecurityManager section in repository.xml is empty
>
>
> Key: OAK-4832
> URL: https://issues.apache.org/jira/browse/OAK-4832
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: upgrade
>Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
>
> In case that the option SecurityManager section in repository.xml is empty,
> the upgrade [fails with the
> NullPointerException|http://jackrabbit.markmail.org/thread/2hhhrorj2huyy3lo]:
> {noformat}
> Caused by: javax.jcr.RepositoryException: Failed to copy content
>at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:525)
>at
> org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.upgrade(OakUpgrade.java:65)
>at
> org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:53)
>... 1 more
> Caused by: java.lang.NullPointerException
>at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.mapSecurityConfig(RepositoryUpgrade.java:615)
>at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:388)
>... 3 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4832) Upgrade breaks if the SecurityManager section in repository.xml is empty
Tomek Rękawek created OAK-4832:
--
Summary: Upgrade breaks if the SecurityManager section in
repository.xml is empty
Key: OAK-4832
URL: https://issues.apache.org/jira/browse/OAK-4832
Project: Jackrabbit Oak
Issue Type: Bug
Components: upgrade
Affects Versions: 1.5.10, 1.2.19, 1.4.7, 1.0.33
Reporter: Tomek Rękawek
Fix For: 1.6
In case that the option SecurityManager section in repository.xml is empty, the
upgrade [fails with the
NullPointerException|http://jackrabbit.markmail.org/thread/2hhhrorj2huyy3lo]:
{noformat}
Caused by: javax.jcr.RepositoryException: Failed to copy content
at
org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:525)
at
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.upgrade(OakUpgrade.java:65)
at
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:53)
... 1 more
Caused by: java.lang.NullPointerException
at
org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.mapSecurityConfig(RepositoryUpgrade.java:615)
at
org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:388)
... 3 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4832) Upgrade breaks if the SecurityManager section in repository.xml is empty
[
https://issues.apache.org/jira/browse/OAK-4832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509328#comment-15509328
]
Tomek Rękawek commented on OAK-4832:
Fixed for trunk in [r1761702|https://svn.apache.org/r1761702].
> Upgrade breaks if the SecurityManager section in repository.xml is empty
>
>
> Key: OAK-4832
> URL: https://issues.apache.org/jira/browse/OAK-4832
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: upgrade
>Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
>
> In case that the option SecurityManager section in repository.xml is empty,
> the upgrade [fails with the
> NullPointerException|http://jackrabbit.markmail.org/thread/2hhhrorj2huyy3lo]:
> {noformat}
> Caused by: javax.jcr.RepositoryException: Failed to copy content
>at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:525)
>at
> org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.upgrade(OakUpgrade.java:65)
>at
> org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:53)
>... 1 more
> Caused by: java.lang.NullPointerException
>at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.mapSecurityConfig(RepositoryUpgrade.java:615)
>at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:388)
>... 3 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509391#comment-15509391
]
Marcel Reutegger commented on OAK-4796:
---
I like the general approach of this patch and how it leverages existing filter
capabilities in the commit hook phase. On the other hand the patch indeed
doesn't help for the case of external changes. I guess there the only option
really is to persist additional information in the (DocumentNodeStore) journal
entry. However, that wouldn't be sufficient. We'd also have to change the
semantics of CommitInfo for external changes, otherwise we cannot pass
information on an external change. We'd also have to add new event filters to
make use of this information. E.g. a listener must be able to declare the names
of the properties it is interested in, etc.
I think for scalability reasons we need a solution that also works for external
changes. Therefore I tend to prefer the proposal that collects additional
information in the commit hook.
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> # events (in the form of ContentChange Objects) occupy a slot of the queue
> even if the listener is not interested in it - any commit lands on any
> listener's queue. This reduces the capacity of the queue for 'actual' events
> to be delivered. It therefore increases the risk that the queue fills - and
> when full has various consequences such as loosing the CommitInfo etc.
> # each event==ContentChange later on must be evaluated, and for that a diff
> must be calculated. Depending on runtime behavior that diff might be
> expensive if no longer in the cache (documentMk specifically).
> As an improvement, this diffing+filtering could be done at an earlier stage
> already, nearer to the commit, and in case the filter would ignore the event,
> it would not have to be put into the queue at all, thus avoiding occupying a
> slot and later potentially slower diffing.
> The suggestion is to implement this via the following algorithm:
> * During the commit, in a {{Validator}} the listener's filters are evaluated
> - in an as-efficient-as-possible manner (Reason for doing it in a Validator
> is that this doesn't add overhead as oak already goes through all changes for
> other Validators). As a result a _list of potentially affected observers_ is
> added to the {{CommitInfo}} (false positives are fine).
> ** Note that the above adds cost to the commit and must therefore be
> carefully done and measured
> ** One potential measure could be to only do filtering when listener's queues
> are larger than a certain threshold (eg 10)
> * The ChangeProcessor in {{contentChanged}} (in the one created in
> [createObserver|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L224])
> then checks the new commitInfo's _potentially affected observers_ list and
> if it's not in the list, adds a {{NOOP}} token at the end of the queue. If
> there's already a NOOP there, the two are collapsed (this way when a filter
> is not affected it would have a NOOP at the end of the queue). If later on a
> no-NOOP item is added, the NOOP's {{root}} is used as the {{previousRoot}}
> for the newly added {{ContentChange}} obj.
> ** To achieve that, the ContentChange obj is extended to not only have the
> "to" {{root}} pointer, but also the "from" {{previousRoot}} pointer which
> currently is implicitly maintained.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4821) Allow use of Java 7 in Oak 1.4
[ https://issues.apache.org/jira/browse/OAK-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-4821. - Resolution: Fixed Fix Version/s: 1.4.8 > Allow use of Java 7 in Oak 1.4 > -- > > Key: OAK-4821 > URL: https://issues.apache.org/jira/browse/OAK-4821 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Affects Versions: 1.4.7 >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.4.8 > > > (mailing list discussion in progress) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4821) Allow use of Java 7 in Oak 1.4
[ https://issues.apache.org/jira/browse/OAK-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509402#comment-15509402 ] Julian Reschke commented on OAK-4821: - 1.4: [r1761707|http://svn.apache.org/r1761707] > Allow use of Java 7 in Oak 1.4 > -- > > Key: OAK-4821 > URL: https://issues.apache.org/jira/browse/OAK-4821 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Affects Versions: 1.4.7 >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.4.8 > > > (mailing list discussion in progress) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4833) Document storage format changes
Michael Dürig created OAK-4833:
--
Summary: Document storage format changes
Key: OAK-4833
URL: https://issues.apache.org/jira/browse/OAK-4833
Project: Jackrabbit Oak
Issue Type: Documentation
Components: doc, segment-tar
Reporter: Michael Dürig
Assignee: Michael Dürig
This issue serves as collection of all changes to the storage format introduced
with Oak Segment Tar and their impact. Once sufficiently stabilised this
information should serve as basis for the documentation in {{oak-doc}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4833) Document storage format changes
[
https://issues.apache.org/jira/browse/OAK-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4833:
---
Issue Type: Technical task (was: Documentation)
Parent: OAK-4292
> Document storage format changes
> ---
>
> Key: OAK-4833
> URL: https://issues.apache.org/jira/browse/OAK-4833
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: doc, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: documentation
>
> This issue serves as collection of all changes to the storage format
> introduced with Oak Segment Tar and their impact. Once sufficiently
> stabilised this information should serve as basis for the documentation in
> {{oak-doc}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Labels: candidate_oak_1_0 (was: candidate_oak_1_0 candidate_oak_1_2) > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0 > Fix For: 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Fix Version/s: 1.2.20 > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0 > Fix For: 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506491#comment-15506491 ] Julian Reschke edited comment on OAK-4583 at 9/21/16 10:30 AM: --- trunk: [r1761571|http://svn.apache.org/r1761571] 1.4: [r1761691|http://svn.apache.org/r1761691] 1.2: [r1761712|http://svn.apache.org/r1761712] was (Author: reschke): trunk: [r1761571|http://svn.apache.org/r1761571] 1.4: [r1761691|http://svn.apache.org/r1761691] > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0 > Fix For: 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4655) Enable configuring multiple segment nodestore instances in same setup
[
https://issues.apache.org/jira/browse/OAK-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4655:
---
Attachment: OAK-4655.v2.patch
> Enable configuring multiple segment nodestore instances in same setup
> -
>
> Key: OAK-4655
> URL: https://issues.apache.org/jira/browse/OAK-4655
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Stefan Egli
> Fix For: 1.6
>
> Attachments: OAK-4655.v1.patch, OAK-4655.v2.patch
>
>
> With OAK-4369 and OAK-4490 its now possible to configure a new
> SegmentNodeStore to act as secondry nodestore (OAK-4180). Recently for few
> other features we see a requirement to configure a SegmentNodeStore just for
> storage purpose. For e.g.
> # OAK-4180 - Enables use of SegmentNodeStore as a secondary store to
> compliment DocumentNodeStore
> #* Always uses BlobStore from primary DocumentNodeStore
> #* Compaction to be enabled
> # OAK-4654 - Enable use of SegmentNodeStore for private mount in a
> multiplexing nodestore setup
> #* Might use its own blob store
> #* Compaction might be disabled as it would be read only
> # OAK-4581 - Proposes to make use of SegmentNodeStore for storing event queue
> offline
> In all these setups we need to configure a SegmentNodeStore which has
> following aspect
> # NodeStore instance is not directly exposed but exposed via
> {{NodeStoreProvider}} interface with {{role}} service property specifying the
> intended usage
> # NodeStore here is not fully functional i.e. it would not be configured with
> std observers, would not be used by ContentRepository etc
> # It needs to be ensured that any JMX MBean registered accounts for "role" so
> that there is no collision
> With existing SegmentNodeStoreService we can only configure 1 nodestore. To
> support above cases we need a OSGi config factory based implementation which
> enables creation of multiple SegmentNodeStore instances (each with different
> directory and different settings)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Labels: (was: candidate_oak_1_0) > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.0.34, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Fix Version/s: 1.0.34 > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.0.34, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506491#comment-15506491 ] Julian Reschke edited comment on OAK-4583 at 9/21/16 11:15 AM: --- trunk: [r1761571|http://svn.apache.org/r1761571] 1.4: [r1761691|http://svn.apache.org/r1761691] 1.2: [r1761712|http://svn.apache.org/r1761712] 1.0: [r1761719|http://svn.apache.org/r1761719] was (Author: reschke): trunk: [r1761571|http://svn.apache.org/r1761571] 1.4: [r1761691|http://svn.apache.org/r1761691] 1.2: [r1761712|http://svn.apache.org/r1761712] > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.0.34, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4834) Make the role configurable for the SegmentNodeStore
Tomek Rękawek created OAK-4834:
--
Summary: Make the role configurable for the SegmentNodeStore
Key: OAK-4834
URL: https://issues.apache.org/jira/browse/OAK-4834
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segment-tar, segmentmk
Reporter: Tomek Rękawek
Priority: Minor
Fix For: 1.6
Right now the SegmentNodeStoreService can be configured with the secondary
boolean property. It'll be then registered as NodeStoreProvider with
{{role=secondary}}.
Maybe we can make it more flexible and allow to configure any role, using
nsProvider.role property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4834:
---
Description:
Right now the SegmentNodeStoreService can be configured with the secondary
boolean property. It'll be then registered as NodeStoreProvider with
{{role=secondary}}.
Maybe we can make it more flexible and allow to configure any role, using
{{nsProvider.role}} property?
was:
Right now the SegmentNodeStoreService can be configured with the secondary
boolean property. It'll be then registered as NodeStoreProvider with
{{role=secondary}}.
Maybe we can make it more flexible and allow to configure any role, using
nsProvider.role property?
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek reassigned OAK-4834:
--
Assignee: Tomek Rękawek
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4655) Enable configuring multiple segment nodestore instances in same setup
[
https://issues.apache.org/jira/browse/OAK-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509608#comment-15509608
]
Tomek Rękawek commented on OAK-4655:
I attached a [slightly modified|^OAK-4655.v2.patch] version that allows to set
any {{role}} for the NodeStoreProvider (using {{nsProvider.role}} property).
It's consistent with my proposal to update SegmentNodeStoreService in OAK-4834.
[~egli], could you take a look on the change and say if you are ok with it? If
so, I'll merge the patch, this feature would be very useful in many different
areas of Oak. It also disabled by default, so I guess it's quiet safe to have
it committed.
> Enable configuring multiple segment nodestore instances in same setup
> -
>
> Key: OAK-4655
> URL: https://issues.apache.org/jira/browse/OAK-4655
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Stefan Egli
> Fix For: 1.6
>
> Attachments: OAK-4655.v1.patch, OAK-4655.v2.patch
>
>
> With OAK-4369 and OAK-4490 its now possible to configure a new
> SegmentNodeStore to act as secondry nodestore (OAK-4180). Recently for few
> other features we see a requirement to configure a SegmentNodeStore just for
> storage purpose. For e.g.
> # OAK-4180 - Enables use of SegmentNodeStore as a secondary store to
> compliment DocumentNodeStore
> #* Always uses BlobStore from primary DocumentNodeStore
> #* Compaction to be enabled
> # OAK-4654 - Enable use of SegmentNodeStore for private mount in a
> multiplexing nodestore setup
> #* Might use its own blob store
> #* Compaction might be disabled as it would be read only
> # OAK-4581 - Proposes to make use of SegmentNodeStore for storing event queue
> offline
> In all these setups we need to configure a SegmentNodeStore which has
> following aspect
> # NodeStore instance is not directly exposed but exposed via
> {{NodeStoreProvider}} interface with {{role}} service property specifying the
> intended usage
> # NodeStore here is not fully functional i.e. it would not be configured with
> std observers, would not be used by ContentRepository etc
> # It needs to be ensured that any JMX MBean registered accounts for "role" so
> that there is no collision
> With existing SegmentNodeStoreService we can only configure 1 nodestore. To
> support above cases we need a OSGi config factory based implementation which
> enables creation of multiple SegmentNodeStore instances (each with different
> directory and different settings)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4834:
---
Attachment: OAK-4834.patch
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
> Attachments: OAK-4834.patch
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509617#comment-15509617
]
Tomek Rękawek commented on OAK-4834:
I attached the patch that modifies the old SegmentNodeStoreService and also
introduces the same change for the segment-tar's one.
[~chetanm], could you take a look?
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
> Attachments: OAK-4834.patch
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4655) Enable configuring multiple segment nodestore instances in same setup
[
https://issues.apache.org/jira/browse/OAK-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli reassigned OAK-4655:
Assignee: Tomek Rękawek (was: Stefan Egli)
[~tomek.rekawek], looks good, more generic than my version which had
'observation' hardcoded. Am assigning the ticket to you then, thx!
> Enable configuring multiple segment nodestore instances in same setup
> -
>
> Key: OAK-4655
> URL: https://issues.apache.org/jira/browse/OAK-4655
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4655.v1.patch, OAK-4655.v2.patch
>
>
> With OAK-4369 and OAK-4490 its now possible to configure a new
> SegmentNodeStore to act as secondry nodestore (OAK-4180). Recently for few
> other features we see a requirement to configure a SegmentNodeStore just for
> storage purpose. For e.g.
> # OAK-4180 - Enables use of SegmentNodeStore as a secondary store to
> compliment DocumentNodeStore
> #* Always uses BlobStore from primary DocumentNodeStore
> #* Compaction to be enabled
> # OAK-4654 - Enable use of SegmentNodeStore for private mount in a
> multiplexing nodestore setup
> #* Might use its own blob store
> #* Compaction might be disabled as it would be read only
> # OAK-4581 - Proposes to make use of SegmentNodeStore for storing event queue
> offline
> In all these setups we need to configure a SegmentNodeStore which has
> following aspect
> # NodeStore instance is not directly exposed but exposed via
> {{NodeStoreProvider}} interface with {{role}} service property specifying the
> intended usage
> # NodeStore here is not fully functional i.e. it would not be configured with
> std observers, would not be used by ContentRepository etc
> # It needs to be ensured that any JMX MBean registered accounts for "role" so
> that there is no collision
> With existing SegmentNodeStoreService we can only configure 1 nodestore. To
> support above cases we need a OSGi config factory based implementation which
> enables creation of multiple SegmentNodeStore instances (each with different
> directory and different settings)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4581) Persistent local journal for more reliable event generation
[
https://issues.apache.org/jira/browse/OAK-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509752#comment-15509752
]
Marcel Reutegger commented on OAK-4581:
---
My preferred approach is I-B-1 with something similar to III-C, structured like
a log (Kafka is probably overkill).
In other issues we so far looked at how to improve the producer side of the
observation system, but even if the producer side is perfect, the queue could
still fill up when the listener is too slow. And once the listener is too slow,
it may have a ripple effect on the producer side. The node state comparisons
may not be served from cache anymore and require I/O interfering/competing with
other read operations in the repository. I think it's best to create the JCR
events as soon as possible and then push them into a queue that is consumed by
the listener. The contents of this queue will obviously have bigger space
requirements than the current queue with just the root states, but it is
simpler to handle. The entries are independent of the repository (unrelated to
GC) and read and write patterns are simple linear operations.
> Persistent local journal for more reliable event generation
> ---
>
> Key: OAK-4581
> URL: https://issues.apache.org/jira/browse/OAK-4581
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4581.v0.patch
>
>
> As discussed in OAK-2683 "hitting the observation queue limit" has multiple
> drawbacks. Quite a bit of work is done to make diff generation faster.
> However there are still chances of event queue getting filled up.
> This issue is meant to implement a persistent event journal. Idea here being
> # NodeStore would push the diff into a persistent store via a synchronous
> observer
> # Observors which are meant to handle such events in async way (by virtue of
> being wrapped in BackgroundObserver) would instead pull the events from this
> persisted journal
> h3. A - What is persisted
> h4. 1 - Serialized Root States and CommitInfo
> In this approach we just persist the root states in serialized form.
> * DocumentNodeStore - This means storing the root revision vector
> * SegmentNodeStore - {color:red}Q1 - What does serialized form of
> SegmentNodeStore root state looks like{color} - Possible the RecordId of
> "root" state
> Note that with OAK-4528 DocumentNodeStore can rely on persisted remote
> journal to determine the affected paths. Which reduces the need for
> persisting complete diff locally.
> Event generation logic would then "deserialize" the persisted root states and
> then generate the diff as currently done via NodeState comparison
> h4. 2 - Serialized commit diff and CommitInfo
> In this approach we can save the diff in JSOP form. The diff only contains
> information about affected path. Similar to what is current being stored in
> DocumentNodeStore journal
> h4. CommitInfo
> The commit info would also need to be serialized. So it needs to be ensure
> whatever is stored there can be serialized or re calculated
> h3. B - How it is persisted
> h4. 1 - Use a secondary segment NodeStore
> OAK-4180 makes use of SegmentNodeStore as a secondary store for caching.
> [~mreutegg] suggested that for persisted local journal we can also utilize a
> SegmentNodeStore instance. Care needs to be taken for compaction. Either via
> generation approach or relying on online compaction
> h4. 2- Make use of write ahead log implementations
> [~ianeboston] suggested that we can make use of some write ahead log
> implementation like [1], [2] or [3]
> h3. C - How changes get pulled
> Some points to consider for event generation logic
> # Would need a way to keep pointers to journal entry on per listener basis.
> This would allow each Listener to "pull" content changes and generate diff as
> per its speed and keeping in memory overhead low
> # The journal should survive restarts
> [1] http://www.mapdb.org/javadoc/latest/mapdb/org/mapdb/WriteAheadLog.html
> [2]
> https://github.com/apache/activemq/tree/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal
> [3]
> https://github.com/elastic/elasticsearch/tree/master/core/src/main/java/org/elasticsearch/index/translog
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4655) Enable configuring multiple segment nodestore instances in same setup
[
https://issues.apache.org/jira/browse/OAK-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509777#comment-15509777
]
Tomek Rękawek commented on OAK-4655:
Fixed for trunk in [r1761723|https://svn.apache.org/r1761723].
> Enable configuring multiple segment nodestore instances in same setup
> -
>
> Key: OAK-4655
> URL: https://issues.apache.org/jira/browse/OAK-4655
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4655.v1.patch, OAK-4655.v2.patch
>
>
> With OAK-4369 and OAK-4490 its now possible to configure a new
> SegmentNodeStore to act as secondry nodestore (OAK-4180). Recently for few
> other features we see a requirement to configure a SegmentNodeStore just for
> storage purpose. For e.g.
> # OAK-4180 - Enables use of SegmentNodeStore as a secondary store to
> compliment DocumentNodeStore
> #* Always uses BlobStore from primary DocumentNodeStore
> #* Compaction to be enabled
> # OAK-4654 - Enable use of SegmentNodeStore for private mount in a
> multiplexing nodestore setup
> #* Might use its own blob store
> #* Compaction might be disabled as it would be read only
> # OAK-4581 - Proposes to make use of SegmentNodeStore for storing event queue
> offline
> In all these setups we need to configure a SegmentNodeStore which has
> following aspect
> # NodeStore instance is not directly exposed but exposed via
> {{NodeStoreProvider}} interface with {{role}} service property specifying the
> intended usage
> # NodeStore here is not fully functional i.e. it would not be configured with
> std observers, would not be used by ContentRepository etc
> # It needs to be ensured that any JMX MBean registered accounts for "role" so
> that there is no collision
> With existing SegmentNodeStoreService we can only configure 1 nodestore. To
> support above cases we need a OSGi config factory based implementation which
> enables creation of multiple SegmentNodeStore instances (each with different
> directory and different settings)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4655) Enable configuring multiple segment nodestore instances in same setup
[
https://issues.apache.org/jira/browse/OAK-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4655:
---
Fix Version/s: 1.5.11
> Enable configuring multiple segment nodestore instances in same setup
> -
>
> Key: OAK-4655
> URL: https://issues.apache.org/jira/browse/OAK-4655
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4655.v1.patch, OAK-4655.v2.patch
>
>
> With OAK-4369 and OAK-4490 its now possible to configure a new
> SegmentNodeStore to act as secondry nodestore (OAK-4180). Recently for few
> other features we see a requirement to configure a SegmentNodeStore just for
> storage purpose. For e.g.
> # OAK-4180 - Enables use of SegmentNodeStore as a secondary store to
> compliment DocumentNodeStore
> #* Always uses BlobStore from primary DocumentNodeStore
> #* Compaction to be enabled
> # OAK-4654 - Enable use of SegmentNodeStore for private mount in a
> multiplexing nodestore setup
> #* Might use its own blob store
> #* Compaction might be disabled as it would be read only
> # OAK-4581 - Proposes to make use of SegmentNodeStore for storing event queue
> offline
> In all these setups we need to configure a SegmentNodeStore which has
> following aspect
> # NodeStore instance is not directly exposed but exposed via
> {{NodeStoreProvider}} interface with {{role}} service property specifying the
> intended usage
> # NodeStore here is not fully functional i.e. it would not be configured with
> std observers, would not be used by ContentRepository etc
> # It needs to be ensured that any JMX MBean registered accounts for "role" so
> that there is no collision
> With existing SegmentNodeStoreService we can only configure 1 nodestore. To
> support above cases we need a OSGi config factory based implementation which
> enables creation of multiple SegmentNodeStore instances (each with different
> directory and different settings)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4655) Enable configuring multiple segment nodestore instances in same setup
[
https://issues.apache.org/jira/browse/OAK-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek resolved OAK-4655.
Resolution: Fixed
> Enable configuring multiple segment nodestore instances in same setup
> -
>
> Key: OAK-4655
> URL: https://issues.apache.org/jira/browse/OAK-4655
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4655.v1.patch, OAK-4655.v2.patch
>
>
> With OAK-4369 and OAK-4490 its now possible to configure a new
> SegmentNodeStore to act as secondry nodestore (OAK-4180). Recently for few
> other features we see a requirement to configure a SegmentNodeStore just for
> storage purpose. For e.g.
> # OAK-4180 - Enables use of SegmentNodeStore as a secondary store to
> compliment DocumentNodeStore
> #* Always uses BlobStore from primary DocumentNodeStore
> #* Compaction to be enabled
> # OAK-4654 - Enable use of SegmentNodeStore for private mount in a
> multiplexing nodestore setup
> #* Might use its own blob store
> #* Compaction might be disabled as it would be read only
> # OAK-4581 - Proposes to make use of SegmentNodeStore for storing event queue
> offline
> In all these setups we need to configure a SegmentNodeStore which has
> following aspect
> # NodeStore instance is not directly exposed but exposed via
> {{NodeStoreProvider}} interface with {{role}} service property specifying the
> intended usage
> # NodeStore here is not fully functional i.e. it would not be configured with
> std observers, would not be used by ContentRepository etc
> # It needs to be ensured that any JMX MBean registered accounts for "role" so
> that there is no collision
> With existing SegmentNodeStoreService we can only configure 1 nodestore. To
> support above cases we need a OSGi config factory based implementation which
> enables creation of multiple SegmentNodeStore instances (each with different
> directory and different settings)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509780#comment-15509780
]
Tomek Rękawek commented on OAK-4834:
BTW, OAK-4655 uses the same convention (nsProvider.role) for the
factory-generated segment node store providers.
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
> Attachments: OAK-4834.patch
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4655) Enable configuring multiple segment nodestore instances in same setup
[
https://issues.apache.org/jira/browse/OAK-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509812#comment-15509812
]
Chetan Mehrotra commented on OAK-4655:
--
[~tomek.rekawek] Would suggest some changes here
* directory - Component should not assume any default here. Or have default
based on role name otherwise they would step on each other
* For "secondary" role I would set {{customBlobStore}} to be true to ensure
that it used the BlobStore used by DocumentNodeStore
* May be some basic test coverage
> Enable configuring multiple segment nodestore instances in same setup
> -
>
> Key: OAK-4655
> URL: https://issues.apache.org/jira/browse/OAK-4655
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4655.v1.patch, OAK-4655.v2.patch
>
>
> With OAK-4369 and OAK-4490 its now possible to configure a new
> SegmentNodeStore to act as secondry nodestore (OAK-4180). Recently for few
> other features we see a requirement to configure a SegmentNodeStore just for
> storage purpose. For e.g.
> # OAK-4180 - Enables use of SegmentNodeStore as a secondary store to
> compliment DocumentNodeStore
> #* Always uses BlobStore from primary DocumentNodeStore
> #* Compaction to be enabled
> # OAK-4654 - Enable use of SegmentNodeStore for private mount in a
> multiplexing nodestore setup
> #* Might use its own blob store
> #* Compaction might be disabled as it would be read only
> # OAK-4581 - Proposes to make use of SegmentNodeStore for storing event queue
> offline
> In all these setups we need to configure a SegmentNodeStore which has
> following aspect
> # NodeStore instance is not directly exposed but exposed via
> {{NodeStoreProvider}} interface with {{role}} service property specifying the
> intended usage
> # NodeStore here is not fully functional i.e. it would not be configured with
> std observers, would not be used by ContentRepository etc
> # It needs to be ensured that any JMX MBean registered accounts for "role" so
> that there is no collision
> With existing SegmentNodeStoreService we can only configure 1 nodestore. To
> support above cases we need a OSGi config factory based implementation which
> enables creation of multiple SegmentNodeStore instances (each with different
> directory and different settings)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4765) Provide option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-4765: - Component/s: (was: segmentmk) > Provide option to interrupt online revision cleanup > --- > > Key: OAK-4765 > URL: https://issues.apache.org/jira/browse/OAK-4765 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > Fix For: Segment Tar 0.0.14 > > Attachments: OAK-4765-v0.patch, OAK-4765-v1.patch > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4835) Provide option to interrupt online revision cleanup on segmentmk
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-4835: - Fix Version/s: (was: Segment Tar 0.0.14) > Provide option to interrupt online revision cleanup on segmentmk > > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4835) Provide option to interrupt online revision cleanup on segmentmk
[ https://issues.apache.org/jira/browse/OAK-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-4835: - Component/s: (was: segment-tar) > Provide option to interrupt online revision cleanup on segmentmk > > > Key: OAK-4835 > URL: https://issues.apache.org/jira/browse/OAK-4835 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4835) Provide option to interrupt online revision cleanup on segmentmk
Alex Parvulescu created OAK-4835: Summary: Provide option to interrupt online revision cleanup on segmentmk Key: OAK-4835 URL: https://issues.apache.org/jira/browse/OAK-4835 Project: Jackrabbit Oak Issue Type: Improvement Components: segment-tar, segmentmk Reporter: Alex Parvulescu Assignee: Alex Parvulescu Fix For: Segment Tar 0.0.14 JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4834:
---
Attachment: OAK-4834.patch
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
> Attachments: OAK-4834.patch
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4834:
---
Attachment: (was: OAK-4834.patch)
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
> Attachments: OAK-4834.patch
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4765) Provide option to interrupt online revision cleanup
[
https://issues.apache.org/jira/browse/OAK-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu resolved OAK-4765.
--
Resolution: Fixed
fixed with http://svn.apache.org/viewvc?rev=1761726&view=rev
I've split the {{segmentmk}} requirements in a clone of this issue, so the 2
can evolve separately as the modules have different release cycles.
> Provide option to interrupt online revision cleanup
> ---
>
> Key: OAK-4765
> URL: https://issues.apache.org/jira/browse/OAK-4765
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
> Labels: compaction, gc
> Fix For: Segment Tar 0.0.14
>
> Attachments: OAK-4765-v0.patch, OAK-4765-v1.patch
>
>
> JMX binding for stopping a running compaction process
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4581) Persistent local journal for more reliable event generation
[
https://issues.apache.org/jira/browse/OAK-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15509914#comment-15509914
]
Stefan Egli commented on OAK-4581:
--
One additional note: should we choose to go the I-B route (queue in
ChangeProcessor), then this improvement will not become usable by Sling's
ResourceChangeListener - as that has switched to using an OakResourceListener
(based on Oak's recommendation to do so, see SLING-3279) and directly bases on
BackgroundObserver...
> Persistent local journal for more reliable event generation
> ---
>
> Key: OAK-4581
> URL: https://issues.apache.org/jira/browse/OAK-4581
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4581.v0.patch
>
>
> As discussed in OAK-2683 "hitting the observation queue limit" has multiple
> drawbacks. Quite a bit of work is done to make diff generation faster.
> However there are still chances of event queue getting filled up.
> This issue is meant to implement a persistent event journal. Idea here being
> # NodeStore would push the diff into a persistent store via a synchronous
> observer
> # Observors which are meant to handle such events in async way (by virtue of
> being wrapped in BackgroundObserver) would instead pull the events from this
> persisted journal
> h3. A - What is persisted
> h4. 1 - Serialized Root States and CommitInfo
> In this approach we just persist the root states in serialized form.
> * DocumentNodeStore - This means storing the root revision vector
> * SegmentNodeStore - {color:red}Q1 - What does serialized form of
> SegmentNodeStore root state looks like{color} - Possible the RecordId of
> "root" state
> Note that with OAK-4528 DocumentNodeStore can rely on persisted remote
> journal to determine the affected paths. Which reduces the need for
> persisting complete diff locally.
> Event generation logic would then "deserialize" the persisted root states and
> then generate the diff as currently done via NodeState comparison
> h4. 2 - Serialized commit diff and CommitInfo
> In this approach we can save the diff in JSOP form. The diff only contains
> information about affected path. Similar to what is current being stored in
> DocumentNodeStore journal
> h4. CommitInfo
> The commit info would also need to be serialized. So it needs to be ensure
> whatever is stored there can be serialized or re calculated
> h3. B - How it is persisted
> h4. 1 - Use a secondary segment NodeStore
> OAK-4180 makes use of SegmentNodeStore as a secondary store for caching.
> [~mreutegg] suggested that for persisted local journal we can also utilize a
> SegmentNodeStore instance. Care needs to be taken for compaction. Either via
> generation approach or relying on online compaction
> h4. 2- Make use of write ahead log implementations
> [~ianeboston] suggested that we can make use of some write ahead log
> implementation like [1], [2] or [3]
> h3. C - How changes get pulled
> Some points to consider for event generation logic
> # Would need a way to keep pointers to journal entry on per listener basis.
> This would allow each Listener to "pull" content changes and generate diff as
> per its speed and keeping in memory overhead low
> # The journal should survive restarts
> [1] http://www.mapdb.org/javadoc/latest/mapdb/org/mapdb/WriteAheadLog.html
> [2]
> https://github.com/apache/activemq/tree/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/journal
> [3]
> https://github.com/elastic/elasticsearch/tree/master/core/src/main/java/org/elasticsearch/index/translog
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4833) Document storage format changes
[
https://issues.apache.org/jira/browse/OAK-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4833:
---
Description:
This issue serves as collection of all changes to the storage format introduced
with Oak Segment Tar and their impact. Once sufficiently stabilised this
information should serve as basis for the documentation in {{oak-doc}}.
was:This issue serves as collection of all changes to the storage format
introduced with Oak Segment Tar and their impact. Once sufficiently stabilised
this information should serve as basis for the documentation in {{oak-doc}}.
> Document storage format changes
> ---
>
> Key: OAK-4833
> URL: https://issues.apache.org/jira/browse/OAK-4833
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: doc, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: documentation
>
> This issue serves as collection of all changes to the storage format
> introduced with Oak Segment Tar and their impact. Once sufficiently
> stabilised this information should serve as basis for the documentation in
> {{oak-doc}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4834:
---
Attachment: (was: OAK-4834.patch)
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
> Attachments: OAK-4834.patch
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4834) Make the role configurable for the SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4834:
---
Attachment: OAK-4834.patch
> Make the role configurable for the SegmentNodeStore
> ---
>
> Key: OAK-4834
> URL: https://issues.apache.org/jira/browse/OAK-4834
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6
>
> Attachments: OAK-4834.patch
>
>
> Right now the SegmentNodeStoreService can be configured with the secondary
> boolean property. It'll be then registered as NodeStoreProvider with
> {{role=secondary}}.
> Maybe we can make it more flexible and allow to configure any role, using
> {{nsProvider.role}} property?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4833) Document storage format changes
[
https://issues.apache.org/jira/browse/OAK-4833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4833:
---
Description:
This issue serves as collection of all changes to the storage format introduced
with Oak Segment Tar and their impact. Once sufficiently stabilised this
information should serve as basis for the documentation in {{oak-doc}}.
|| Change || Rational || Impact || Migration || Since || Issues ||
|Generation in segment header |Required to unequivocally determine the
generation of a segment during cleanup. Segment retention time is given in
number of generations (2 by default). |No performance, space impact expected
|offline |0.0.2 |OAK-3348 |
|Stable id for node states |Required to efficiently determine equality of node
states. This can be seen as an intermediate step to decoupling the address of
records from their identity. The next step is to introduce logical record ids
(OAK-4659). |Node states increase by the size of one record id (3 bytes / 20
bytes after OAK-4631). On top of that there is an additional block record à 18
bytes per node state. |offline |0.0.2 |OAK-3348
|Binary index in tar files |Avoid traversing the repository to collect the gc
roots for DSGC. Fetch them from an index instead. |Additional index entry per
tar file. Adds a couple of bytes per external binary to each tar file. Exact
size to be determined. [~frm] could you help with this? OAK-4740 is a
regression wrt. to resiliency caused by this change (and the fact that the blob
store might return blob ids longer than 2k chars). |offline |0.0.4 |OAK-4101
|Simplified record ids |Preparation and precondition for logical record ids
(OAK-4659). At the same time the simplest possible fix for OAK-2896. The latter
leads to degeneration of segment sizes, which in turn has adverse effects on
overall performance, resource utilisation and memory requirements. Without this
fix OAK-2498 would need to be fixed in a different way that would require other
changes in the storage format. I started to regard this issue as removing a
premature optimisation (which caused OAK-2498). |Record ids grow from 3 bytes
to 18 bytes when serialised into records. Impact on repositories to be assessed
but can be anywhere between almost none to x6. OAK-4812 is a performance
regression caused by this chance. Its overall impact is yet to be assessed.
|offline |0.0.10 |OAK-4631
|Storage format versioning |In order to be able to further evolve the storage
format with minimal impact on existing deployments we need to carefully
versions the various storage entities (segments, tar files, etc.) |No
performance, space impact expected |offline |0.0.2/ 0.0.10 |OAK-4232, OAK-4683,
OAK-4295
|Logical record ids |We need to separate addresses of records from their
identity to be able to further scale the TarMK. OAK-3348 (the online compaction
misery) can be seen as a symptom of failing to understand this earlier. The
stable ids introduced with OAK-3348 are a first step into this direction.
However this is not sufficient to implement features like e.g. background
compaction (OAK-4756), partial compaction (OAK-3349) or incremental compaction
(OAK-3350). |A small size overhead per segment for the logical id table.
Further impact to be evaluated ([~frm], please add your assessment here).
|offline |0.0.14 (planned) |OAK-4659
|External index for segments |Avoid recreating tar files if indexes are
corrupt/missing. Just recreate the indexes. |Faster startup after a crash.
Overall less disk space usage as no unnecessary backup files are created.
|online |not yet planned |OAK-4649
|In-place journal |Reduce complexity by in-lining the journal log. Less files,
less chances to break something. Also the granularity of the log would increase
as flushing of the persisted head would not be required any more. Resilience
would improve as the roll-back functionality could operate at a finer
granularity. |No more journal.log. Better resiliency. Significant risk for
regression of OAK-4291 if not implemented properly. Most likely a significant
refactoring of some parts of the code is required before we can proceed with
this issue. |online |not yet planned |OAK-4103
|Root record types |With the information currently available from the segment
headers we cannot collect statistics about segment usage on repositories of non
trivial sizes. This fix would allow us to build more scalable tools to that
respect. |None expected wrt. to performance and size under normal operation.
|offline |0.0.14 (planned) (waiting for OAK-4659 as implementation depends on
how we progress there) |OAK-2498
Misc ideas currently on the back burner:
* SegmentMK: Arch segments (OAK-1905)
* Extension headers for segments (no issue yet)
* More memory efficient serialisation of values (e.g. boolean) (no issue yet)
* Protocol Buffer for serialising records (no issue yet)
was:
Th
[jira] [Created] (OAK-4836) Avoid excessive logging in case of corrupt or mis-configured index defnitino
Vikas Saurabh created OAK-4836: -- Summary: Avoid excessive logging in case of corrupt or mis-configured index defnitino Key: OAK-4836 URL: https://issues.apache.org/jira/browse/OAK-4836 Project: Jackrabbit Oak Issue Type: Improvement Components: lucene Reporter: Vikas Saurabh Assignee: Vikas Saurabh Priority: Minor Following up from comment\[0] from [~chetanm] in OAK-4805, in case of corrupt of mis-configured index defnition the logs tend to fill up with same exception over and over. It would be useful for index tracker to keep track of such index and ignore those for any processing until some change is detected on the definition (reindex would also lead to a change on index). We might also want to expose a jmx which lists the indices that tracker thinks are bad (and probably also why it started to think that). \[0]: https://issues.apache.org/jira/browse/OAK-4805?focusedCommentId=15492487&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15492487 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510194#comment-15510194 ] Vikas Saurabh commented on OAK-4805: I haven't got time to work on the issue so opened OAK-4836 for avoiding filling up logs. Would commit the patch soon (with additional change of IAE to Exception). > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > Fix For: 1.6 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Saurabh resolved OAK-4805. Resolution: Fixed Fix Version/s: 1.5.11 Fixed in trunk at [r1761762|https://svn.apache.org/r1761762]. > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > Fix For: 1.6, 1.5.11 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4836) Avoid excessive logging in case of corrupt or mis-configured index defnition
[ https://issues.apache.org/jira/browse/OAK-4836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Saurabh updated OAK-4836: --- Summary: Avoid excessive logging in case of corrupt or mis-configured index defnition (was: Avoid excessive logging in case of corrupt or mis-configured index defnitino) > Avoid excessive logging in case of corrupt or mis-configured index defnition > > > Key: OAK-4836 > URL: https://issues.apache.org/jira/browse/OAK-4836 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh >Priority: Minor > > Following up from comment\[0] from [~chetanm] in OAK-4805, in case of corrupt > of mis-configured index defnition the logs tend to fill up with same > exception over and over. It would be useful for index tracker to keep track > of such index and ignore those for any processing until some change is > detected on the definition (reindex would also lead to a change on index). > We might also want to expose a jmx which lists the indices that tracker > thinks are bad (and probably also why it started to think that). > \[0]: > https://issues.apache.org/jira/browse/OAK-4805?focusedCommentId=15492487&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15492487 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4836) Avoid excessive logging in case of corrupt index or mis-configured index defnition
[ https://issues.apache.org/jira/browse/OAK-4836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Saurabh updated OAK-4836: --- Summary: Avoid excessive logging in case of corrupt index or mis-configured index defnition (was: Avoid excessive logging in case of corrupt or mis-configured index defnition) > Avoid excessive logging in case of corrupt index or mis-configured index > defnition > -- > > Key: OAK-4836 > URL: https://issues.apache.org/jira/browse/OAK-4836 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh >Priority: Minor > > Following up from comment\[0] from [~chetanm] in OAK-4805, in case of corrupt > of mis-configured index defnition the logs tend to fill up with same > exception over and over. It would be useful for index tracker to keep track > of such index and ignore those for any processing until some change is > detected on the definition (reindex would also lead to a change on index). > We might also want to expose a jmx which lists the indices that tracker > thinks are bad (and probably also why it started to think that). > \[0]: > https://issues.apache.org/jira/browse/OAK-4805?focusedCommentId=15492487&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15492487 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4836) Avoid excessive logging in case of corrupt index or mis-configured index defnition
[ https://issues.apache.org/jira/browse/OAK-4836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Saurabh updated OAK-4836: --- Description: Following up from comment\[0] from [~chetanm] in OAK-4805, in case of corrupt index or mis-configured index defnition the logs tend to fill up with same exception over and over. It would be useful for index tracker to keep track of such index and ignore those for any processing until some change is detected on the definition (reindex would also lead to a change on index). We might also want to expose a jmx which lists the indices that tracker thinks are bad (and probably also why it started to think that). \[0]: https://issues.apache.org/jira/browse/OAK-4805?focusedCommentId=15492487&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15492487 was: Following up from comment\[0] from [~chetanm] in OAK-4805, in case of corrupt of mis-configured index defnition the logs tend to fill up with same exception over and over. It would be useful for index tracker to keep track of such index and ignore those for any processing until some change is detected on the definition (reindex would also lead to a change on index). We might also want to expose a jmx which lists the indices that tracker thinks are bad (and probably also why it started to think that). \[0]: https://issues.apache.org/jira/browse/OAK-4805?focusedCommentId=15492487&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15492487 > Avoid excessive logging in case of corrupt index or mis-configured index > defnition > -- > > Key: OAK-4836 > URL: https://issues.apache.org/jira/browse/OAK-4836 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh >Priority: Minor > > Following up from comment\[0] from [~chetanm] in OAK-4805, in case of corrupt > index or mis-configured index defnition the logs tend to fill up with same > exception over and over. It would be useful for index tracker to keep track > of such index and ignore those for any processing until some change is > detected on the definition (reindex would also lead to a change on index). > We might also want to expose a jmx which lists the indices that tracker > thinks are bad (and probably also why it started to think that). > \[0]: > https://issues.apache.org/jira/browse/OAK-4805?focusedCommentId=15492487&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15492487 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4813) Simplify the server side of cold standby
[
https://issues.apache.org/jira/browse/OAK-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-4813:
-
Attachment: OAK-4813-01.patch
This comes hand in hand with OAK-4803 and aims to simplify the
{{StandbyServer}} code by moving all JMX related stuff into a new class,
{{StandbyServerSync}}. Other important changes
* Removed {{FileStore}} reference from {{StandbyServer}}. The fileStore is now
acquired using a {{StoreProvider}}, currently implemented by
{{StandbyServerSync}}
* Updated {{StandbyServer}} and {{StandbyClient}} to use the latest
{{SslContextBuilder}} from netty in favour of old deprecated methods
[~mduerig], could you please review this patch, please?
/cc [~alexparvulescu], [~frm]
> Simplify the server side of cold standby
>
>
> Key: OAK-4813
> URL: https://issues.apache.org/jira/browse/OAK-4813
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Fix For: Segment Tar 0.0.14
>
> Attachments: OAK-4813-01.patch
>
>
> With the changes introduced in OAK-4803, it would be nice to keep the
> previous symmetry between the client and server and remove thus the
> {{FileStore}} reference from the latter.
> Per [~frm]'s suggestion from one of the comments in OAK-4803:
> bq. In the end, these are the only three lines where the FileStore is used in
> the server, which already suggests that this separation of concerns exists -
> at least at the level of the handlers.
> {code:java}
> p.addLast(new GetHeadRequestHandler(new DefaultStandbyHeadReader(store)));
> p.addLast(new GetSegmentRequestHandler(new
> DefaultStandbySegmentReader(store)));
> p.addLast(new GetBlobRequestHandler(new DefaultStandbyBlobReader(store)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4813) Simplify the server side of cold standby
[
https://issues.apache.org/jira/browse/OAK-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510213#comment-15510213
]
Andrei Dulceanu edited comment on OAK-4813 at 9/21/16 3:06 PM:
---
This comes hand in hand with OAK-4803 and aims to simplify the
{{StandbyServer}} code by moving all JMX related stuff into a new class,
{{StandbyServerSync}}. Other important changes
* Removed {{FileStore}} reference from {{StandbyServer}}. The fileStore is now
acquired using a {{StoreProvider}}, currently implemented by
{{StandbyServerSync}}
* Updated {{StandbyServer}} and {{StandbyClient}} to use the latest
{{SslContextBuilder}} from netty in favour of old deprecated methods
[~mduerig], could you review this patch, please?
/cc [~alexparvulescu], [~frm]
was (Author: dulceanu):
This comes hand in hand with OAK-4803 and aims to simplify the
{{StandbyServer}} code by moving all JMX related stuff into a new class,
{{StandbyServerSync}}. Other important changes
* Removed {{FileStore}} reference from {{StandbyServer}}. The fileStore is now
acquired using a {{StoreProvider}}, currently implemented by
{{StandbyServerSync}}
* Updated {{StandbyServer}} and {{StandbyClient}} to use the latest
{{SslContextBuilder}} from netty in favour of old deprecated methods
[~mduerig], could you please review this patch, please?
/cc [~alexparvulescu], [~frm]
> Simplify the server side of cold standby
>
>
> Key: OAK-4813
> URL: https://issues.apache.org/jira/browse/OAK-4813
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Fix For: Segment Tar 0.0.14
>
> Attachments: OAK-4813-01.patch
>
>
> With the changes introduced in OAK-4803, it would be nice to keep the
> previous symmetry between the client and server and remove thus the
> {{FileStore}} reference from the latter.
> Per [~frm]'s suggestion from one of the comments in OAK-4803:
> bq. In the end, these are the only three lines where the FileStore is used in
> the server, which already suggests that this separation of concerns exists -
> at least at the level of the handlers.
> {code:java}
> p.addLast(new GetHeadRequestHandler(new DefaultStandbyHeadReader(store)));
> p.addLast(new GetSegmentRequestHandler(new
> DefaultStandbySegmentReader(store)));
> p.addLast(new GetBlobRequestHandler(new DefaultStandbyBlobReader(store)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4283) Align GCMonitor API with implementation
[
https://issues.apache.org/jira/browse/OAK-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu reassigned OAK-4283:
Assignee: Andrei Dulceanu (was: Francesco Mari)
> Align GCMonitor API with implementation
>
>
> Key: OAK-4283
> URL: https://issues.apache.org/jira/browse/OAK-4283
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
> Labels: api-change, compaction, gc
> Fix For: Segment Tar 0.0.16
>
>
> The argument taken by {{GCMonitor.compacted}} related to parameters of the
> compaction map. The latter has gone with OAK-3348. We need to come up with a
> way to adjust this API accordingly. Also it might make sense to broaden the
> scope of {{GCMonitor}} from its initial intent (logging) to a more general
> one as this is how it is already used e.g. by the {{RefreshOnGC}}
> implementation and for OAK-4096.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510201#comment-15510201 ] Vikas Saurabh edited comment on OAK-4805 at 9/21/16 3:23 PM: - Fixed in trunk at [r1761762|https://svn.apache.org/r1761762]. Backported to 1.4 at [r1761768|https://svn.apache.org/r1761768] and 1.2 at [r1761769|https://svn.apache.org/r1761769]. was (Author: catholicon): Fixed in trunk at [r1761762|https://svn.apache.org/r1761762]. > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > Fix For: 1.6, 1.5.11 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Saurabh updated OAK-4805: --- Labels: candidate_oak_1_0 (was: candidate_oak_1_0 candidate_oak_1_2 candidate_oak_1_4) > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0 > Fix For: 1.6, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Saurabh updated OAK-4805: --- Fix Version/s: 1.4.8 > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0 > Fix For: 1.6, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Saurabh updated OAK-4805: --- Fix Version/s: 1.2.20 > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0 > Fix For: 1.6, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510283#comment-15510283 ] Vikas Saurabh commented on OAK-4805: [~chetanm], I'd like to backport this to 1.0 as well, but in absence of custom analyzer feature in 1.0 I don't know how to test a mis-configured index. Ideas? > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0 > Fix For: 1.6, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4742) Improve FileStoreStatsMBean
[
https://issues.apache.org/jira/browse/OAK-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510292#comment-15510292
]
Michael Dürig commented on OAK-4742:
We also need to align this with the information exposed via {{GCMonitorMBean}}.
> Improve FileStoreStatsMBean
> ---
>
> Key: OAK-4742
> URL: https://issues.apache.org/jira/browse/OAK-4742
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: monitoring
> Fix For: Segment Tar 0.0.16
>
>
> We should add further data to that MBean (if feasible):
> * Number of commits
> * Number of commits queuing (blocked on the commit semaphore)
> * Percentiles of commit times (exclude queueing time)
> * Percentiles of commit queueing times
> * Last gc run / size before gc and after gc / time gc took broken down into
> the various phases
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4689) Add information about amount of data vs. waste to oak-run
[
https://issues.apache.org/jira/browse/OAK-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-4689:
--
Assignee: Michael Dürig
> Add information about amount of data vs. waste to oak-run
> -
>
> Key: OAK-4689
> URL: https://issues.apache.org/jira/browse/OAK-4689
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: run
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: production, tooling
> Fix For: 1.6
>
>
> This is a follow up for OAK-3695, where we decided that it would be to
> expensive doing this as live monitoring.
> Instead we should add functionality to {{oak-run}} that could connect in read
> only mode to a running repository and collect information about how much data
> and how must wast the repository contains.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3695) Expose ratio between waste and real data
[ https://issues.apache.org/jira/browse/OAK-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-3695. Resolution: Won't Fix Resolving as wont fix. As discussed the desired functionality will be provided via OAK-4689 and OAK-4742. > Expose ratio between waste and real data > > > Key: OAK-3695 > URL: https://issues.apache.org/jira/browse/OAK-3695 > Project: Jackrabbit Oak > Issue Type: Story > Components: segment-tar >Reporter: Valentin Olteanu > Labels: gc > Fix For: Segment Tar 0.0.16 > > > As a user, I want to know the ratio (or precise absolute values) between > waste and real data on TarMK, so that I can decide if Revision GC needs to be > run. The measurement has to be done on a running repository and without > impacting the performance. > This would also help measure the efficiency of Revision GC and see the effect > of improvements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4742) Improve FileStoreStatsMBean
[ https://issues.apache.org/jira/browse/OAK-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4742: --- Description: We should add further data to that MBean (if feasible): * Number of commits * Number of commits queuing (blocked on the commit semaphore) * Percentiles of commit times (exclude queueing time) * Percentiles of commit queueing times * Last gc run / size before gc and after gc / time gc took broken down into the various phases was: We should add further data to that MBean (if feasible): * Number of commits * Number of commits queuing (blocked on the commit semaphore) * Percentiles of commit times (exclude queueing time) * Percentiles of commit queueing times > Improve FileStoreStatsMBean > --- > > Key: OAK-4742 > URL: https://issues.apache.org/jira/browse/OAK-4742 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Michael Dürig > Labels: monitoring > Fix For: Segment Tar 0.0.16 > > > We should add further data to that MBean (if feasible): > * Number of commits > * Number of commits queuing (blocked on the commit semaphore) > * Percentiles of commit times (exclude queueing time) > * Percentiles of commit queueing times > * Last gc run / size before gc and after gc / time gc took broken down into > the various phases -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-3695) Expose ratio between waste and real data
[
https://issues.apache.org/jira/browse/OAK-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15432370#comment-15432370
]
Michael Dürig edited comment on OAK-3695 at 9/21/16 3:31 PM:
-
bq. Customers might want to have some statistics about it in order to decide
their maintenance schedule
For this customers are probably better off with the information provided by
{{GCMonitorMBean}} as that one provides information on how much the last gc run
could actually retain and how big the repository was afterwards. We probably
should also add information regarding duration of the last gc run.-[~volteanu]
if you could think of other information that would be useful to add here please
open a issue for that.- I think we should add this information to the
{{FileStoreStatsMBean}}. See OAK-4742.
bq. now run in read-only mode
Yes, read only.
was (Author: mduerig):
bq. Customers might want to have some statistics about it in order to decide
their maintenance schedule
For this customers are probably better off with the information provided by
{{GCMonitorMBean}} as that one provides information on how much the last gc run
could actually retain and how big the repository was afterwards. We probably
should also add information regarding duration of the last gc run. [~volteanu]
if you could think of other information that would be useful to add here please
open a issue for that.
bq. now run in read-only mode
Yes, read only.
> Expose ratio between waste and real data
>
>
> Key: OAK-3695
> URL: https://issues.apache.org/jira/browse/OAK-3695
> Project: Jackrabbit Oak
> Issue Type: Story
> Components: segment-tar
>Reporter: Valentin Olteanu
> Labels: gc
> Fix For: Segment Tar 0.0.16
>
>
> As a user, I want to know the ratio (or precise absolute values) between
> waste and real data on TarMK, so that I can decide if Revision GC needs to be
> run. The measurement has to be done on a running repository and without
> impacting the performance.
> This would also help measure the efficiency of Revision GC and see the effect
> of improvements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4813) Simplify the server side of cold standby
[
https://issues.apache.org/jira/browse/OAK-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510372#comment-15510372
]
Michael Dürig commented on OAK-4813:
Being completely new to the standby implementation this might not apply at all.
But I'm not sure whether this patch addresses what [~frm] had in mind. AFAICS
{{StandByServer}} still has a dependency to the {{FileStore}}. Only that it is
now looped through {{builder.storeProvider.provideStore()}}. To make the server
truly symmetric to the client some sort of decoupling between fetching the data
from the store and sending it from the server needs to be implemented. The
client does this via the various blocking queues. However, I can't really judge
whether this is what [~frm] was thinking of and whether this is good or bad.
Probably best to wait for him to have a look and clarify.
> Simplify the server side of cold standby
>
>
> Key: OAK-4813
> URL: https://issues.apache.org/jira/browse/OAK-4813
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Fix For: Segment Tar 0.0.14
>
> Attachments: OAK-4813-01.patch
>
>
> With the changes introduced in OAK-4803, it would be nice to keep the
> previous symmetry between the client and server and remove thus the
> {{FileStore}} reference from the latter.
> Per [~frm]'s suggestion from one of the comments in OAK-4803:
> bq. In the end, these are the only three lines where the FileStore is used in
> the server, which already suggests that this separation of concerns exists -
> at least at the level of the handlers.
> {code:java}
> p.addLast(new GetHeadRequestHandler(new DefaultStandbyHeadReader(store)));
> p.addLast(new GetSegmentRequestHandler(new
> DefaultStandbySegmentReader(store)));
> p.addLast(new GetBlobRequestHandler(new DefaultStandbyBlobReader(store)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4371) Overly zealous warning about checkpoints on compaction
[
https://issues.apache.org/jira/browse/OAK-4371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510390#comment-15510390
]
Michael Dürig commented on OAK-4371:
[~alex.parvulescu] as you worked on OAK-4043, would you have an idea how to fix
this. Anything we can do on our side without introducing hard to maintain
implementation dependencies to the async indexing? Otherwise I suggest to
remove this check and recommend external monitoring as suggested in my previous
comment.
> Overly zealous warning about checkpoints on compaction
> ---
>
> Key: OAK-4371
> URL: https://issues.apache.org/jira/browse/OAK-4371
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Reporter: Michael Dürig
> Labels: compaction, gc, logging
> Fix For: Segment Tar 0.0.14
>
>
> {{FileStore.compact}} logs a warning {{TarMK GC #{}: compaction found {}
> checkpoints, you might need to run checkpoint cleanup}} if there is more than
> a single checkpoints.
> AFIK this is now the norm as async indexing has uses 2 checkpoints
> ([~chetanm], [~edivad] please clarify).
> In any case should we improve this and not hard code any number of expected
> checkpoints. Maybe make the threshold configurable?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4122) Replace the commit semaphore in the segment node store with a scheduler
[
https://issues.apache.org/jira/browse/OAK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4122:
---
Fix Version/s: (was: Segment Tar 0.0.16)
> Replace the commit semaphore in the segment node store with a scheduler
> ---
>
> Key: OAK-4122
> URL: https://issues.apache.org/jira/browse/OAK-4122
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
> Labels: performance, scalability, throughput
>
> {{SegmentNodeStore}} currently uses a semaphore to coordinate concurrent
> commits thus relying on the scheduling algorithm of that implementation and
> ultimately of the JVM for in what order commits are processed.
> I think it would be beneficial to replace that semaphore with an explicit
> queue of pending commit. This would allow us to implement a proper scheduler
> optimising for e.g. minimal system load, maximal throughput or minimal
> latency etc. A scheduler could e.g. give precedence to big commits and order
> commits along the order of its base revisions, which would decrease the
> amount of work to be done in rebasing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4122) Replace the commit semaphore in the segment node store with a scheduler
[
https://issues.apache.org/jira/browse/OAK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510398#comment-15510398
]
Michael Dürig commented on OAK-4122:
Unscheduling this for now as with OAK-4015 we have a solution for the most
imminent problem. I suggest we take this up again after Oak 1.6
> Replace the commit semaphore in the segment node store with a scheduler
> ---
>
> Key: OAK-4122
> URL: https://issues.apache.org/jira/browse/OAK-4122
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
> Labels: performance, scalability, throughput
>
> {{SegmentNodeStore}} currently uses a semaphore to coordinate concurrent
> commits thus relying on the scheduling algorithm of that implementation and
> ultimately of the JVM for in what order commits are processed.
> I think it would be beneficial to replace that semaphore with an explicit
> queue of pending commit. This would allow us to implement a proper scheduler
> optimising for e.g. minimal system load, maximal throughput or minimal
> latency etc. A scheduler could e.g. give precedence to big commits and order
> commits along the order of its base revisions, which would decrease the
> amount of work to be done in rebasing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[
https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510201#comment-15510201
]
Vikas Saurabh edited comment on OAK-4805 at 9/21/16 6:25 PM:
-
Fixed in trunk at [r1761762|https://svn.apache.org/r1761762]. Backported to 1.4
at [r1761768|https://svn.apache.org/r1761768] and 1.2 at
[r1761769|https://svn.apache.org/r1761769].
Fixed broken test ({{LuceneIndexTest#indexNodeLockHandling}}) at
[r1761787|https://svn.apache.org/r1761787] on trunk, at
[r1761791|https://svn.apache.org/r1761791] on 1.4 and at
[r1761792|https://svn.apache.org/r1761792] on 1.2. I had run the tests with
{{IllegalArgumentException}} and assumed that generalizing to {{Exception}} was
ok... sorry for the noise :-(.
was (Author: catholicon):
Fixed in trunk at [r1761762|https://svn.apache.org/r1761762]. Backported to 1.4
at [r1761768|https://svn.apache.org/r1761768] and 1.2 at
[r1761769|https://svn.apache.org/r1761769].
> Misconfigured lucene index definition can render the whole system unusable
> --
>
> Key: OAK-4805
> URL: https://issues.apache.org/jira/browse/OAK-4805
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Labels: candidate_oak_1_0
> Fix For: 1.6, 1.4.8, 1.5.11, 1.2.20
>
> Attachments: OAK-4805.patch
>
>
> Mis-configured index definition can throw an exception while collecting
> plans. This causes any query (even unrelated ones) to not work as cost
> calculation logic would consult a badly constructed index def. Overall a
> mis-configured index definition can practically grind the whole system to
> halt as the whole query framework stops working.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4805) Misconfigured lucene index definition can render the whole system unusable
[ https://issues.apache.org/jira/browse/OAK-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510283#comment-15510283 ] Vikas Saurabh edited comment on OAK-4805 at 9/21/16 6:26 PM: - [~chetanm], I'd like to backport this to 1.0 as well, but in absence of custom analyzer feature in 1.0 I don't know how to test a mis-configured index. Ideas? Btw, I broke the build with that last commit which kind of implies that it testable on 1.0. was (Author: catholicon): [~chetanm], I'd like to backport this to 1.0 as well, but in absence of custom analyzer feature in 1.0 I don't know how to test a mis-configured index. Ideas? > Misconfigured lucene index definition can render the whole system unusable > -- > > Key: OAK-4805 > URL: https://issues.apache.org/jira/browse/OAK-4805 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene >Reporter: Vikas Saurabh >Assignee: Vikas Saurabh > Labels: candidate_oak_1_0 > Fix For: 1.6, 1.4.8, 1.5.11, 1.2.20 > > Attachments: OAK-4805.patch > > > Mis-configured index definition can throw an exception while collecting > plans. This causes any query (even unrelated ones) to not work as cost > calculation logic would consult a badly constructed index def. Overall a > mis-configured index definition can practically grind the whole system to > halt as the whole query framework stops working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4655) Enable configuring multiple segment nodestore instances in same setup
[
https://issues.apache.org/jira/browse/OAK-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15510939#comment-15510939
]
Tomek Rękawek commented on OAK-4655:
[~chetanm], thanks for the suggestions. I applied all of them in
[r1761799|https://svn.apache.org/r1761799].
What about the SegmentNodeStoreService? Do you think we should make it
consistent (as in OAK-4834) or once we have the factory we don't need the
NodeStoreProvider capabilities in the "main" service?
> Enable configuring multiple segment nodestore instances in same setup
> -
>
> Key: OAK-4655
> URL: https://issues.apache.org/jira/browse/OAK-4655
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: segment-tar, segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4655.v1.patch, OAK-4655.v2.patch
>
>
> With OAK-4369 and OAK-4490 its now possible to configure a new
> SegmentNodeStore to act as secondry nodestore (OAK-4180). Recently for few
> other features we see a requirement to configure a SegmentNodeStore just for
> storage purpose. For e.g.
> # OAK-4180 - Enables use of SegmentNodeStore as a secondary store to
> compliment DocumentNodeStore
> #* Always uses BlobStore from primary DocumentNodeStore
> #* Compaction to be enabled
> # OAK-4654 - Enable use of SegmentNodeStore for private mount in a
> multiplexing nodestore setup
> #* Might use its own blob store
> #* Compaction might be disabled as it would be read only
> # OAK-4581 - Proposes to make use of SegmentNodeStore for storing event queue
> offline
> In all these setups we need to configure a SegmentNodeStore which has
> following aspect
> # NodeStore instance is not directly exposed but exposed via
> {{NodeStoreProvider}} interface with {{role}} service property specifying the
> intended usage
> # NodeStore here is not fully functional i.e. it would not be configured with
> std observers, would not be used by ContentRepository etc
> # It needs to be ensured that any JMX MBean registered accounts for "role" so
> that there is no collision
> With existing SegmentNodeStoreService we can only configure 1 nodestore. To
> support above cases we need a OSGi config factory based implementation which
> enables creation of multiple SegmentNodeStore instances (each with different
> directory and different settings)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
[
https://issues.apache.org/jira/browse/OAK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Klimetschek updated OAK-4825:
---
Attachment: OAK-4825-b.patch
Attached [new patch|^OAK-4825-b.patch] (alternatively [on
github|https://github.com/alexkli/jackrabbit-oak/commit/6f2b0455f2cae2a69c2be8981e15e143d3d55011].
Improvements:
* introduce {{SyncResult.Status.ENABLE}} and {{DISABLE}} for more accurate
status reporting (it is already detailed, so I figured this new feature should
be covered)
* one caveat: if a user is re-enabled and updated along the way, you get
{{ENABLE}} and not {{UPDATE}}
* only re-enable users if the {{users.disableMissing}} config is {{true}}; the
at this point unlikely case of systems migrating from disable to the remove
option would have to be handled by "manually" enabling affected users
* added a test that ensures this, i.e. that if the default remove option is on,
the disabled status of users is not touched
> Support disabling of users instead of removal in DefaultSyncHandler
> ---
>
> Key: OAK-4825
> URL: https://issues.apache.org/jira/browse/OAK-4825
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: auth-external
>Reporter: Alexander Klimetschek
> Attachments: OAK-4825-b.patch, OAK-4825.patch
>
>
> The DefaultSyncHandler by default will remove (local) users when they are no
> longer active in the external system aka no longer provided by the
> ExternalIdentityProvider. It would be useful to have an option to _disable_
> users instead of removing them completely.
> This is good for use cases that need to keep the user's data around in the
> JCR and can't just delete it. Also, we have seen cases where the user is only
> temporarily removed from the external identity system (e.g. accidentally
> removed from group that maps them to the JCR system and quickly added back),
> where a full removal can do harm.
> (Note: There is an [option in the SyncContext
> interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38]
> to suppress purging, and the JMX sync commands such as
> [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256]
> "use" it. However, the JCR users look like "valid" users then locally. Even
> if the authentication is done completely through the IDP and will fail
> properly for these missing users, it can be difficult for other uses like
> administration, monitoring, other application code to detect that such a user
> is not active anymore.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
[
https://issues.apache.org/jira/browse/OAK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15511479#comment-15511479
]
Alexander Klimetschek edited comment on OAK-4825 at 9/21/16 11:07 PM:
--
Attached [new patch|^OAK-4825-b.patch] (alternatively [on
github|https://github.com/alexkli/jackrabbit-oak/commit/6f2b0455f2cae2a69c2be8981e15e143d3d55011]).
Improvements:
* introduce {{SyncResult.Status.ENABLE}} and {{DISABLE}} for more accurate
status reporting (it is already detailed, so I figured this new feature should
be covered)
* one caveat: if a user is re-enabled and updated along the way, you get
{{ENABLE}} and not {{UPDATE}}
* only re-enable users if the {{users.disableMissing}} config is {{true}}; the
at this point unlikely case of systems migrating from disable to the remove
option would have to be handled by "manually" enabling affected users
* added a test that ensures this, i.e. that if the default remove option is on,
the disabled status of users is not touched
was (Author: alexander.klimetschek):
Attached [new patch|^OAK-4825-b.patch] (alternatively [on
github|https://github.com/alexkli/jackrabbit-oak/commit/6f2b0455f2cae2a69c2be8981e15e143d3d55011].
Improvements:
* introduce {{SyncResult.Status.ENABLE}} and {{DISABLE}} for more accurate
status reporting (it is already detailed, so I figured this new feature should
be covered)
* one caveat: if a user is re-enabled and updated along the way, you get
{{ENABLE}} and not {{UPDATE}}
* only re-enable users if the {{users.disableMissing}} config is {{true}}; the
at this point unlikely case of systems migrating from disable to the remove
option would have to be handled by "manually" enabling affected users
* added a test that ensures this, i.e. that if the default remove option is on,
the disabled status of users is not touched
> Support disabling of users instead of removal in DefaultSyncHandler
> ---
>
> Key: OAK-4825
> URL: https://issues.apache.org/jira/browse/OAK-4825
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: auth-external
>Reporter: Alexander Klimetschek
> Attachments: OAK-4825-b.patch, OAK-4825.patch
>
>
> The DefaultSyncHandler by default will remove (local) users when they are no
> longer active in the external system aka no longer provided by the
> ExternalIdentityProvider. It would be useful to have an option to _disable_
> users instead of removing them completely.
> This is good for use cases that need to keep the user's data around in the
> JCR and can't just delete it. Also, we have seen cases where the user is only
> temporarily removed from the external identity system (e.g. accidentally
> removed from group that maps them to the JCR system and quickly added back),
> where a full removal can do harm.
> (Note: There is an [option in the SyncContext
> interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38]
> to suppress purging, and the JMX sync commands such as
> [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256]
> "use" it. However, the JCR users look like "valid" users then locally. Even
> if the authentication is done completely through the IDP and will fail
> properly for these missing users, it can be difficult for other uses like
> administration, monitoring, other application code to detect that such a user
> is not active anymore.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4837) Improved caching for DataStore
Amit Jain created OAK-4837:
--
Summary: Improved caching for DataStore
Key: OAK-4837
URL: https://issues.apache.org/jira/browse/OAK-4837
Project: Jackrabbit Oak
Issue Type: Improvement
Components: blob
Reporter: Amit Jain
Assignee: Amit Jain
The current CachingDataStore implementation used with S3DataStore has certain
problems:
* Lack of stats to show hit rate/miss rates for files being requested for
downloads
* Lack of stats for async uploads
* Async upload functionality leaks into Backend implementations, LocalCache
classes.
* The call to {{DataStore#getRecord()}} which makes multiple calls to backends
which is problematic for S3 (i.e. when not being served bu cache)
* There is some functionality which is not used with Oak like length cache,
sync/async touch etc. which can be removed and code simplified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4838) Move S3 classes to oak-blob-cloud module
Amit Jain created OAK-4838: -- Summary: Move S3 classes to oak-blob-cloud module Key: OAK-4838 URL: https://issues.apache.org/jira/browse/OAK-4838 Project: Jackrabbit Oak Issue Type: Technical task Components: blob Reporter: Amit Jain Assignee: Amit Jain Some S3 related classes are present in oak-core module. These should be moved to oak-blob-cloud. This would also flip the module dependencies to oak-core -> oak-blob-cloud -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4838) Move S3 classes to oak-blob-cloud module
[ https://issues.apache.org/jira/browse/OAK-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain resolved OAK-4838. Resolution: Fixed Fix Version/s: 1.5.11 Done with http://svn.apache.org/viewvc?rev=1761852&view=rev > Move S3 classes to oak-blob-cloud module > > > Key: OAK-4838 > URL: https://issues.apache.org/jira/browse/OAK-4838 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain > Labels: technical_debt > Fix For: 1.5.11 > > > Some S3 related classes are present in oak-core module. These should be moved > to oak-blob-cloud. > This would also flip the module dependencies to oak-core -> oak-blob-cloud -- This message was sent by Atlassian JIRA (v6.3.4#6332)
