[jira] [Resolved] (OAK-7172) Document TarMK specific MBeans
[
https://issues.apache.org/jira/browse/OAK-7172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-7172.
Resolution: Done
Added basic documentation at
[http://svn.apache.org/viewvc?rev=1821681=rev] and
http://svn.apache.org/viewvc?rev=1821682=rev.
> Document TarMK specific MBeans
> --
>
> Key: OAK-7172
> URL: https://issues.apache.org/jira/browse/OAK-7172
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: doc, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.1
>
>
> Currently the [TarMK
> documentation|http://jackrabbit.apache.org/oak/docs/nodestore/segment/overview.html#monitoring-via-jmx]
> only mentions {{SegmentRevisionGarbageCollection}}. We should review that
> paragraph and also include documentation for all other relevant JMX endpoints.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (OAK-6964) Document tail compaction
[ https://issues.apache.org/jira/browse/OAK-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-6964. Resolution: Done Done at: * [http://svn.apache.org/viewvc?rev=1821665=rev] * [http://svn.apache.org/viewvc?rev=1821668=rev] And back ported to 1.8 at: * [http://svn.apache.org/viewvc?rev=1821669=rev] * [http://svn.apache.org/viewvc?rev=1821670=rev] * [http://svn.apache.org/viewvc?rev=1821671=rev] > Document tail compaction > > > Key: OAK-6964 > URL: https://issues.apache.org/jira/browse/OAK-6964 > Project: Jackrabbit Oak > Issue Type: Documentation > Components: doc, segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig >Priority: Major > Labels: documentation > Fix For: 1.9.0, 1.10, 1.8.1 > > > We need to add documentation of tail compaction: > * What is it, how does it work? > * How is it configured and scheduled? > * How can it be monitored, what are the related log entries? > * What are its limitations? > * What if it fails? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7181) RDBDocumentStore: use try-with-resources for ChangesTracker
[ https://issues.apache.org/jira/browse/OAK-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-7181: Issue Type: Technical task (was: Task) Parent: OAK-1266 > RDBDocumentStore: use try-with-resources for ChangesTracker > --- > > Key: OAK-7181 > URL: https://issues.apache.org/jira/browse/OAK-7181 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Labels: candidate_oak_1_8 > Fix For: 1.9.0, 1.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7181) RDBDocumentStore: use try-with-resources for ChangesTracker
[ https://issues.apache.org/jira/browse/OAK-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332347#comment-16332347 ] Julian Reschke commented on OAK-7181: - trunk: [r1821663|http://svn.apache.org/r1821663] > RDBDocumentStore: use try-with-resources for ChangesTracker > --- > > Key: OAK-7181 > URL: https://issues.apache.org/jira/browse/OAK-7181 > Project: Jackrabbit Oak > Issue Type: Task > Components: rdbmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Labels: candidate_oak_1_8 > Fix For: 1.9.0, 1.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7181) RDBDocumentStore: use try-with-resources for ChangesTracker
[ https://issues.apache.org/jira/browse/OAK-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-7181: Labels: candidate_oak_1_8 (was: ) > RDBDocumentStore: use try-with-resources for ChangesTracker > --- > > Key: OAK-7181 > URL: https://issues.apache.org/jira/browse/OAK-7181 > Project: Jackrabbit Oak > Issue Type: Task > Components: rdbmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Labels: candidate_oak_1_8 > Fix For: 1.9.0, 1.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-7181) RDBDocumentStore: use try-with-resources for ChangesTracker
[ https://issues.apache.org/jira/browse/OAK-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-7181. - Resolution: Fixed Fix Version/s: 1.10 1.9.0 > RDBDocumentStore: use try-with-resources for ChangesTracker > --- > > Key: OAK-7181 > URL: https://issues.apache.org/jira/browse/OAK-7181 > Project: Jackrabbit Oak > Issue Type: Task > Components: rdbmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.9.0, 1.10 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-2849) Improve revision gc on SegmentMK
[ https://issues.apache.org/jira/browse/OAK-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2849: --- Epic Status: Done > Improve revision gc on SegmentMK > > > Key: OAK-2849 > URL: https://issues.apache.org/jira/browse/OAK-2849 > Project: Jackrabbit Oak > Issue Type: Epic > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig >Priority: Major > Labels: compaction, gc > Fix For: 1.5.13, 1.6.0 > > Attachments: SegmentCompactionIT-conflicts.png > > > This is a container issue for the ongoing effort to improve revision gc of > the SegmentMK. > I'm exploring > * ways to make the reference graph as exact as possible and necessary: it > should not contain segments that are not referenceable any more and but must > contain all segments that are referenceable. > * ways to segregate the reference graph reducing dependencies between certain > set of segments as much as possible. > * Reducing the number of in memory references and their impact on gc as much > as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-4243) Oak Segment Tar Module
[
https://issues.apache.org/jira/browse/OAK-4243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4243:
---
Epic Status: Done (was: To Do)
> Oak Segment Tar Module
> --
>
> Key: OAK-4243
> URL: https://issues.apache.org/jira/browse/OAK-4243
> Project: Jackrabbit Oak
> Issue Type: Epic
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Fix For: 1.6.1, 1.8.0
>
>
> There is a couple of issues requiring us to change the segment format in a
> non compatible way (OAK-3348, OAK-2896, OAK-2498, OAK-4201).
> We should introduce a new module here
> * to minimise ripple effect on concurrent development work in other parts of
> Oak and upstream projects
> * to be able to cleanly migrate existing repositories via a side grading
> * to cleanly separate breaking changes from the existing code base
> The plan is roughly to:
> * Create new module (called {{oak-segment-next}} for now, will discuss names
> later)
> * Apply patch prepared for OAK-3348
> * Discuss and decide on final name and refactor accordingly
> * Reactor affected tooling such that the targeted segment store can be
> specified via an option. Keep default at {{oak-segment}}.
> * Once sufficiently stabilised, deprecate {{oak-segment}}. Make this one the
> default in switch the default target for tooling.
> * Define and implement migration path
> I will create respective issues as needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-5056) Improve GC scalability on TarMK
[ https://issues.apache.org/jira/browse/OAK-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-5056: --- Epic Colour: ghx-label-6 (was: ghx-label-1) > Improve GC scalability on TarMK > --- > > Key: OAK-5056 > URL: https://issues.apache.org/jira/browse/OAK-5056 > Project: Jackrabbit Oak > Issue Type: Epic > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig >Priority: Major > Labels: gc, scalability > Fix For: 1.9.0, 1.10 > > > This issue is about making TarMK gc more scalable: > * how to deal with huge repositories. > * how to deal with massive concurrent writes. > * how can we improve monitoring to determine gc health. > ** Monitor deduplication caches (e.g. deduplication of checkpoints) > Possible avenues to explore: > * Can we partition gc? (e.g. along sub-trees, along volatile vs. static > content) > * Can we pause and resume gc? (e.g. to give precedence to concurrent writes) > * Can we make gc a real background process not contending with foreground > operations? > This issue is a follow up to OAK-2849, which was about efficacy of gc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-5464) Improve the transaction rate of the TarMK
[
https://issues.apache.org/jira/browse/OAK-5464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-5464:
---
Epic Colour: ghx-label-2 (was: ghx-label-3)
> Improve the transaction rate of the TarMK
> -
>
> Key: OAK-5464
> URL: https://issues.apache.org/jira/browse/OAK-5464
> Project: Jackrabbit Oak
> Issue Type: Epic
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: scalability
> Fix For: 1.9.0, 1.10
>
>
> The TarMK's write throughput is limited by the way concurrent commits are
> processed: rebasing and running the commit hooks happen within a lock without
> any explicit scheduling. This epic covers improving the overall transaction
> rate. The proposed approach would roughly be to first make scheduling of
> transactions explicit, then add monitoring on transaction to gather a better
> understanding and then experiment and implement explicit scheduling
> strategies to optimise particular aspects.
> h2. Summary of ideas mentioned in an offline sessions
> h3. Advantages of explicit scheduling:
> * Control over (order) of commits
> * Sophisticated monitoring (commit statistics, e.g. commit rate, time in
> queue, etc.)
> * Favour certain commits (e.g. checkpoints)
> * Reorder commits to simplify rebasing
> * Suspend the compactor on concurrent commits and have it resume where it
> left off afterwards
> * Parallelise certain commits (e.g. by piggy backing)
> * Implement a concurrent commit editor. we'd need to take care of proper
> access to the shared state; [~frm] maybe introduce the idea of a common
> context to enforce concurrent access semantics.
> h3. Scheduler Implementation
> * Expedite
> * Prioritise
> * Defer
> * Collapse
> * Coalesce
> * Parallelise
> * Piggy back: can we piggy back commits on top of each other? The idea would
> be while processing the changes of one commit to also check them for
> conflicts with the changes of other commits waiting to commit. If a conflict
> is detected there, that other commit can immediately be failed (given the
> current commit doesn't fail).
> * Merging non conflicting commits. Given multiple transactions ready to
> commit at the same time. Can we process them as one (given they don't
> conflict) instead of one after each other, which requires rebasing the later
> transaction to be rebase on the former.
> * Shield the file store from {{InterruptedException}} because of thread
> boundaries introduced
> * Implement tests, benchmarks and fixtures for verification
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-3696) Improve SegmentMK resilience
[ https://issues.apache.org/jira/browse/OAK-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-3696: --- Epic Status: Done (was: To Do) > Improve SegmentMK resilience > > > Key: OAK-3696 > URL: https://issues.apache.org/jira/browse/OAK-3696 > Project: Jackrabbit Oak > Issue Type: Epic > Components: segment-tar >Reporter: Michael Marth >Assignee: Michael Dürig >Priority: Major > Labels: resilience > Fix For: Segment Tar 0.0.16 > > > Epic for collection SegmentMK resilience improvements -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-2849) Improve revision gc on SegmentMK
[ https://issues.apache.org/jira/browse/OAK-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2849: --- Epic Colour: ghx-label-2 > Improve revision gc on SegmentMK > > > Key: OAK-2849 > URL: https://issues.apache.org/jira/browse/OAK-2849 > Project: Jackrabbit Oak > Issue Type: Epic > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig >Priority: Major > Labels: compaction, gc > Fix For: 1.5.13, 1.6.0 > > Attachments: SegmentCompactionIT-conflicts.png > > > This is a container issue for the ongoing effort to improve revision gc of > the SegmentMK. > I'm exploring > * ways to make the reference graph as exact as possible and necessary: it > should not contain segments that are not referenceable any more and but must > contain all segments that are referenceable. > * ways to segregate the reference graph reducing dependencies between certain > set of segments as much as possible. > * Reducing the number of in memory references and their impact on gc as much > as possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7169) The datastorecheck returns a zero exit code on error
[ https://issues.apache.org/jira/browse/OAK-7169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-7169: --- Fix Version/s: 1.8.1 > The datastorecheck returns a zero exit code on error > > > Key: OAK-7169 > URL: https://issues.apache.org/jira/browse/OAK-7169 > Project: Jackrabbit Oak > Issue Type: Bug > Components: run >Reporter: Francesco Mari >Assignee: Francesco Mari >Priority: Major > Labels: candidate_oak_1_8 > Fix For: 1.9.0, 1.10, 1.8.1 > > > The datastorecheck command should return a non-zero exit code if it fails > with an exception. Moreover, every error message should be printed on the > standard error. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7174) The check command returns a zero exit code on error
[ https://issues.apache.org/jira/browse/OAK-7174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-7174: --- Fix Version/s: 1.8.1 1.9.0 > The check command returns a zero exit code on error > --- > > Key: OAK-7174 > URL: https://issues.apache.org/jira/browse/OAK-7174 > Project: Jackrabbit Oak > Issue Type: Bug > Components: run, segment-tar >Reporter: Francesco Mari >Assignee: Francesco Mari >Priority: Major > Fix For: 1.9.0, 1.10, 1.8.1 > > > The check command should return a non-zero exit code if it fails with an > exception. Moreover, every error message should be printed on the standard > error. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-6941) Compatibility matrix for oak-run compact
[
https://issues.apache.org/jira/browse/OAK-6941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-6941:
---
Fix Version/s: 1.8.1
> Compatibility matrix for oak-run compact
>
>
> Key: OAK-6941
> URL: https://issues.apache.org/jira/browse/OAK-6941
> Project: Jackrabbit Oak
> Issue Type: Documentation
> Components: doc, run, segment-tar
>Reporter: Valentin Olteanu
>Priority: Major
> Labels: documentation, tooling
> Fix For: 1.9.0, 1.10, 1.8.1
>
>
> h4. Problem statement
> For compacting the segmentstore using {{oak-run}}, the safest option is to
> use the same version of {{oak-run}} as the Oak version used to generate the
> repository. Yet, sometimes, a newer {{oak-run}} version is recommended to
> benefit of bug fixes and improvements, but not every combination of source
> repo and oak-run is safe to use and the user needs a way to check the
> compatibility. Thus, the users need a tool that guides the decision of which
> version to use.
> h4. Requirements
> * Easy to decide what {{oak-run}} version should be used for a certain Oak
> version
> * Up to date with the latest releases
> * Machine readable for scripting
> * Include details on the benefits of using a certain version (release notes)
> * Blacklist of versions that should not be used (with alternatives)
> h4. Solution
> TBD
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-6031) Add TarFiles to the architecture diagram
[
https://issues.apache.org/jira/browse/OAK-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-6031:
---
Fix Version/s: 1.8.1
> Add TarFiles to the architecture diagram
>
>
> Key: OAK-6031
> URL: https://issues.apache.org/jira/browse/OAK-6031
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: doc, segment-tar
>Affects Versions: 1.8.0
>Reporter: Francesco Mari
>Assignee: Francesco Mari
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.1
>
>
> The newly created {{TarFiles}} should be added to the architecture diagram
> for oak-segment-tar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7112) Update documentation for cold standby
[
https://issues.apache.org/jira/browse/OAK-7112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-7112:
---
Fix Version/s: 1.8.1
> Update documentation for cold standby
> -
>
> Key: OAK-7112
> URL: https://issues.apache.org/jira/browse/OAK-7112
> Project: Jackrabbit Oak
> Issue Type: Documentation
> Components: doc, segment-tar, tarmk-standby
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Major
> Fix For: 1.9.0, 1.10, 1.8.1
>
>
> Improve monitoring section of cold standby in {{oak-doc}} to include missing
> MBean screenshots.
> [~mduerig], [~frm]: How about adding a *Benchmarking* section to the cold
> standby page covering a bit ways to use the new {{Oak-Segment-Tar-Cold}}
> fixture and also running {{ScalabilityStandbySuite}} on top of it?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7173) Update documentation for oak-run check
[
https://issues.apache.org/jira/browse/OAK-7173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-7173:
---
Fix Version/s: 1.8.1
> Update documentation for oak-run check
> --
>
> Key: OAK-7173
> URL: https://issues.apache.org/jira/browse/OAK-7173
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: doc, oak-run, segment-tar
>Reporter: Michael Dürig
>Assignee: Andrei Dulceanu
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.1
>
>
> We should review and update the documentation of [{{oak-run
> check}}|http://jackrabbit.apache.org/oak/docs/nodestore/segment/overview.html#check].
> E.g. to include the new options from OAK-6373.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7109) rep:facet returns wrong results for complex queries
[
https://issues.apache.org/jira/browse/OAK-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332289#comment-16332289
]
Dirk Rudolph commented on OAK-7109:
---
Thanks for the response. Regarding 1) see
https://issues.apache.org/jira/browse/OAK-7109?focusedCommentId=16309376=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16309376
Optimisation does the following at the moment:
A and (B or not(C and D)) => (A and B) or (A and not(C and D))
To achieve an optimisation where the result is a DNF, which can then be split
in UNIONS of exclusively conjunctions, another step needs to happen before the
current optimisation - NNF (moving all negation down the tree of statements)
A and (B or not(C or D)) => A and (B or not(C) or not(B)) => (A and B) or (A
and not(C)) or (A and not(B))
Not sure if the index supports not() but if it does, the UNION of the query
above (3) queries would give exact facets which simply need to be deduplicated.
> rep:facet returns wrong results for complex queries
> ---
>
> Key: OAK-7109
> URL: https://issues.apache.org/jira/browse/OAK-7109
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7
>Reporter: Dirk Rudolph
>Priority: Major
> Labels: facet
> Attachments: facetsInMultipleRoots.patch,
> restrictionPropagationTest.patch
>
>
> eComplex queries in that case are queries, which are passed to lucene not
> containing all original constraints. For example queries with multiple path
> restrictions like:
> {code}
> select [rep:facet(simple/tags)] from [nt:base] as a where contains(a.[*],
> 'ipsum') and (isdescendantnode(a,'/content1') or
> isdescendantnode(a,'/content2'))
> {code}
> In that particular case the index planer gives ":fulltext:ipsum" to lucene
> even though the index supports evaluating path constraints.
> As counting the facets happens on the raw result of lucene, the returned
> facets are incorrect. For example having the following content
> {code}
> /content1/test/foo
> + text = lorem ipsum
> - simple/
> + tags = tag1, tag2
> /content2/test/bar
> + text = lorem ipsum
> - simple/
> + tags = tag1, tag2
> /content3/test/bar
> + text = lorem ipsum
> - simple/
>+ tags = tag1, tag2
> {code}
> the expected result for the dimensions of simple/tags and the query above is
> - tag1: 2
> - tag2: 2
> as the result set is 2 results long and all documents are equal. The actual
> result set is
> - tag1: 3
> - tag2: 3
> as the path constraint is not handled by lucene.
> To workaround that the only solution that came to my mind is building the
> [disjunctive normal
> form|https://en.wikipedia.org/wiki/Disjunctive_normal_form] of my complex
> query and executing a query for each of the disjunctive statements. As this
> is expanding exponentially its only a theoretical solution, nothing for
> production.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7172) Document TarMK specific MBeans
[
https://issues.apache.org/jira/browse/OAK-7172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-7172:
---
Fix Version/s: 1.8.1
> Document TarMK specific MBeans
> --
>
> Key: OAK-7172
> URL: https://issues.apache.org/jira/browse/OAK-7172
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: doc, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.1
>
>
> Currently the [TarMK
> documentation|http://jackrabbit.apache.org/oak/docs/nodestore/segment/overview.html#monitoring-via-jmx]
> only mentions {{SegmentRevisionGarbageCollection}}. We should review that
> paragraph and also include documentation for all other relevant JMX endpoints.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-6964) Document tail compaction
[ https://issues.apache.org/jira/browse/OAK-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-6964: --- Fix Version/s: 1.8.1 > Document tail compaction > > > Key: OAK-6964 > URL: https://issues.apache.org/jira/browse/OAK-6964 > Project: Jackrabbit Oak > Issue Type: Documentation > Components: doc, segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig >Priority: Major > Labels: documentation > Fix For: 1.9.0, 1.10, 1.8.1 > > > We need to add documentation of tail compaction: > * What is it, how does it work? > * How is it configured and scheduled? > * How can it be monitored, what are the related log entries? > * What are its limitations? > * What if it fails? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7151) Support indexed based excerpts on properties
[
https://issues.apache.org/jira/browse/OAK-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332281#comment-16332281
]
Vikas Saurabh commented on OAK-7151:
Thanks for the review [~tmueller]. Have committed
[r1821652|https://svn.apache.org/r1821652] and
[r1821653|https://svn.apache.org/r1821653] in trunk. Not resolving just yet
though as I want to check a few other scenarios.
> Support indexed based excerpts on properties
>
>
> Key: OAK-7151
> URL: https://issues.apache.org/jira/browse/OAK-7151
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
>Priority: Major
> Fix For: 1.10
>
> Attachments: OAK-7151.patch, OAK-7151.xpath-new-syntax.patch,
> OAK-7151.xpath.patch
>
>
> As discovered in OAK-4401 we fallback to {{SimpleExcerptProvider}} when
> requesting excerpts for properties.
> The issue as highlighted in [~teofili]'s comment \[0] is that we at time of
> query we don't have information about which all columns/fields would be
> required for excerpts.
> A possible approach is that the query specified explicitly which columns
> would be required in facets (of course, node level excerpt would still be
> supported). This issue is to track that improvement.
> Note: this is *not* a substitute for OAK-4401 which is about doing saner
> highlighting when {{SimpleExcerptProvider}} comes into play e.g. despite this
> issue excerpt for non-stored fields (properties which aren't configured with
> {{useInExcerpt}} in the index definition}, we'd need to fallback to
> {{SimpleExcerptProvider}}.
> /[~tmueller]
> \[0]:
> https://issues.apache.org/jira/browse/OAK-4401?focusedCommentId=15299857=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15299857
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OAK-7075) Document oak-run compact arguments and system properties
[
https://issues.apache.org/jira/browse/OAK-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-7075:
---
Fix Version/s: 1.8.1
> Document oak-run compact arguments and system properties
>
>
> Key: OAK-7075
> URL: https://issues.apache.org/jira/browse/OAK-7075
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: doc, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.1
>
>
> Ensure {{oak-doc}} is up to date with the current version of {{oak-run
> compact}}, its current command line arguments and system properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7075) Document oak-run compact arguments and system properties
[
https://issues.apache.org/jira/browse/OAK-7075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332280#comment-16332280
]
Michael Dürig commented on OAK-7075:
Merged into 1.8 at http://svn.apache.org/viewvc?rev=1821656=rev
> Document oak-run compact arguments and system properties
>
>
> Key: OAK-7075
> URL: https://issues.apache.org/jira/browse/OAK-7075
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: doc, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Major
> Labels: documentation
> Fix For: 1.9.0, 1.10, 1.8.1
>
>
> Ensure {{oak-doc}} is up to date with the current version of {{oak-run
> compact}}, its current command line arguments and system properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7109) rep:facet returns wrong results for complex queries
[
https://issues.apache.org/jira/browse/OAK-7109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332267#comment-16332267
]
Thomas Mueller commented on OAK-7109:
-
[~diru] I'm sorry for the delay. I'm afraid I can't follow you... Some links
(more for myself, please tell me if I made a mistake):
CNF https://en.wikipedia.org/wiki/Conjunctive_normal_form
example: A and (B or C)
DNF https://en.wikipedia.org/wiki/Disjunctive_normal_form
example: (A and not(B) and not((C)) or (not(D) and E and F)
NNF https://en.wikipedia.org/wiki/Negation_normal_form
example: (A or B) and C
> all constraints have to be passed to lucene, so the query has to be in DNF,
> which is not the case at the moment
Only the filter is passed to Lucene currently, and that one doesn't have any
"or" conditions (except for "x in(1, 2, 3)"). Changing that will be hard, and
has some disadvantages. Other "or" conditions are currently only supported by
using "union" (aggregation), or by not processing them in the index (filtering
in the query engine).
So I think it's not so much about "not" conditions.
> would require also a deduplication between the lucene result sets returned
> from each of the unions.
Yes. I think that's possible, even though it's not optimal.
> rep:facet returns wrong results for complex queries
> ---
>
> Key: OAK-7109
> URL: https://issues.apache.org/jira/browse/OAK-7109
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.7
>Reporter: Dirk Rudolph
>Priority: Major
> Labels: facet
> Attachments: facetsInMultipleRoots.patch,
> restrictionPropagationTest.patch
>
>
> eComplex queries in that case are queries, which are passed to lucene not
> containing all original constraints. For example queries with multiple path
> restrictions like:
> {code}
> select [rep:facet(simple/tags)] from [nt:base] as a where contains(a.[*],
> 'ipsum') and (isdescendantnode(a,'/content1') or
> isdescendantnode(a,'/content2'))
> {code}
> In that particular case the index planer gives ":fulltext:ipsum" to lucene
> even though the index supports evaluating path constraints.
> As counting the facets happens on the raw result of lucene, the returned
> facets are incorrect. For example having the following content
> {code}
> /content1/test/foo
> + text = lorem ipsum
> - simple/
> + tags = tag1, tag2
> /content2/test/bar
> + text = lorem ipsum
> - simple/
> + tags = tag1, tag2
> /content3/test/bar
> + text = lorem ipsum
> - simple/
>+ tags = tag1, tag2
> {code}
> the expected result for the dimensions of simple/tags and the query above is
> - tag1: 2
> - tag2: 2
> as the result set is 2 results long and all documents are equal. The actual
> result set is
> - tag1: 3
> - tag2: 3
> as the path constraint is not handled by lucene.
> To workaround that the only solution that came to my mind is building the
> [disjunctive normal
> form|https://en.wikipedia.org/wiki/Disjunctive_normal_form] of my complex
> query and executing a query for each of the disjunctive statements. As this
> is expanding exponentially its only a theoretical solution, nothing for
> production.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-7151) Support indexed based excerpts on properties
[
https://issues.apache.org/jira/browse/OAK-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332237#comment-16332237
]
Thomas Mueller commented on OAK-7151:
-
[~catholicon] looks good to me, except for one typo: "exceprt"
> Support indexed based excerpts on properties
>
>
> Key: OAK-7151
> URL: https://issues.apache.org/jira/browse/OAK-7151
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
>Priority: Major
> Fix For: 1.10
>
> Attachments: OAK-7151.patch, OAK-7151.xpath-new-syntax.patch,
> OAK-7151.xpath.patch
>
>
> As discovered in OAK-4401 we fallback to {{SimpleExcerptProvider}} when
> requesting excerpts for properties.
> The issue as highlighted in [~teofili]'s comment \[0] is that we at time of
> query we don't have information about which all columns/fields would be
> required for excerpts.
> A possible approach is that the query specified explicitly which columns
> would be required in facets (of course, node level excerpt would still be
> supported). This issue is to track that improvement.
> Note: this is *not* a substitute for OAK-4401 which is about doing saner
> highlighting when {{SimpleExcerptProvider}} comes into play e.g. despite this
> issue excerpt for non-stored fields (properties which aren't configured with
> {{useInExcerpt}} in the index definition}, we'd need to fallback to
> {{SimpleExcerptProvider}}.
> /[~tmueller]
> \[0]:
> https://issues.apache.org/jira/browse/OAK-4401?focusedCommentId=15299857=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15299857
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OAK-7181) RDBDocumentStore: use try-with-resources for ChangesTracker
Julian Reschke created OAK-7181: --- Summary: RDBDocumentStore: use try-with-resources for ChangesTracker Key: OAK-7181 URL: https://issues.apache.org/jira/browse/OAK-7181 Project: Jackrabbit Oak Issue Type: Task Components: rdbmk Reporter: Julian Reschke Assignee: Julian Reschke -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OAK-7156) CacheChangesTracker should implement Closeable
[ https://issues.apache.org/jira/browse/OAK-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332094#comment-16332094 ] Julian Reschke commented on OAK-7156: - trunk: [r1821617|http://svn.apache.org/r1821617] > CacheChangesTracker should implement Closeable > -- > > Key: OAK-7156 > URL: https://issues.apache.org/jira/browse/OAK-7156 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > Labels: candidate_oak_1_8 > Attachments: OAK-7156.diff > > > So it can be used with try-with-resources... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7156) CacheChangesTracker should implement Closeable
[ https://issues.apache.org/jira/browse/OAK-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-7156: Labels: candidate_oak_1_8 (was: ) > CacheChangesTracker should implement Closeable > -- > > Key: OAK-7156 > URL: https://issues.apache.org/jira/browse/OAK-7156 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Major > Labels: candidate_oak_1_8 > Attachments: OAK-7156.diff > > > So it can be used with try-with-resources... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OAK-7180) The error message "postings highlighting failed" should be warn or debug
[ https://issues.apache.org/jira/browse/OAK-7180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-7180: Component/s: lucene > The error message "postings highlighting failed" should be warn or debug > > > Key: OAK-7180 > URL: https://issues.apache.org/jira/browse/OAK-7180 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Thomas Mueller >Priority: Major > Fix For: 1.10 > > > We log an error, saying "postings highlighting failed", but a user seeing > that message is confused because: > * the message doesn't say how severe this is > * the code just continues with a fallback, the error message doesn't say that > * it's unclear on how to resolve the issue > Possibly we should just log a warning, or with debug level. Plus extend the > message saying the fallback was used. > It looks like this was introduced in OAK-4368. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OAK-7180) The error message "postings highlighting failed" should be warn or debug
Thomas Mueller created OAK-7180: --- Summary: The error message "postings highlighting failed" should be warn or debug Key: OAK-7180 URL: https://issues.apache.org/jira/browse/OAK-7180 Project: Jackrabbit Oak Issue Type: Improvement Reporter: Thomas Mueller Fix For: 1.10 We log an error, saying "postings highlighting failed", but a user seeing that message is confused because: * the message doesn't say how severe this is * the code just continues with a fallback, the error message doesn't say that * it's unclear on how to resolve the issue Possibly we should just log a warning, or with debug level. Plus extend the message saying the fallback was used. It looks like this was introduced in OAK-4368. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (OAK-6015) ACL of versioned node can be modified without checking out the node
[
https://issues.apache.org/jira/browse/OAK-6015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Deparvu resolved OAK-6015.
---
Resolution: Not A Problem
thanks for the clarifications [~anchela], [~mreutegg]!. it seems this works as
designed, I was also a bit confused while working on the patch.
> ACL of versioned node can be modified without checking out the node
> ---
>
> Key: OAK-6015
> URL: https://issues.apache.org/jira/browse/OAK-6015
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.6.0
>Reporter: Marco Piovesana
>Priority: Major
> Attachments: OAK-6015-v0.patch
>
>
> On a versione node _nodeA_ i can do:
> {{AccessControlUtils.clear(nodeA, userPrincipal)}}
> without having to checkout the node.
> After saving the session I tried to login as _userPrincipal_ and I couldn't
> find _nodeA_, so it seems that the clear operation did work even if the node
> was checked-in.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OAK-6309) Not always convert XPath "primaryType in a, b" to union
[
https://issues.apache.org/jira/browse/OAK-6309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331867#comment-16331867
]
Thomas Mueller commented on OAK-6309:
-
The system properties "oak.xpathNodeTypeOptimization" and
"oak.xpathNodeTypeUnion" already allow to disable this feature.
> Not always convert XPath "primaryType in a, b" to union
> ---
>
> Key: OAK-6309
> URL: https://issues.apache.org/jira/browse/OAK-6309
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Critical
> Fix For: 1.10
>
>
> Currently, queries with multiple primary types are always converted to a
> "union", but this is not alway the best solution. The main problem is that
> results are not sorted by score as expected. Example:
> {noformat}
> /jcr:root/content//element(*, nt:hierarchyNode)[jcr:contains(., 'abc)
> and (@jcr:primaryType = 'acme:Page' or @jcr:primaryType = 'acme:Asset')]
> {noformat}
> This is currently converted to a union, even if the same index is used for
> buth subqueries (assuming there is an index on nt:hierarchyNode).
> A workaround is to use:
> {noformat}
> /jcr:root/content//element(*, nt:hierarchyNode)[jcr:contains(., 'abc)
> and (./@jcr:primaryType = 'acme:Page' or ./@jcr:primaryType = 'acme:Asset')]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
