[jira] [Commented] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195573#comment-16195573 ] Chetan Mehrotra commented on OAK-6787: -- bq. I guess that check would happen at the wiring end though, right? Ack. Would check in LuceneIndexProviderService and configure that in PropertyIndexCleaner at time of init. PropertyIndexCleaner would have a flag exposed for recursiveDelete which would be enabled by LuceneIndexProviderService upon determining the type of NodeStore > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6787-v1.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6795) Build Jackrabbit Oak #844 failed
[ https://issues.apache.org/jira/browse/OAK-6795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195545#comment-16195545 ] Hudson commented on OAK-6795: - Build is still failing. Failed run: [Jackrabbit Oak #845|https://builds.apache.org/job/Jackrabbit%20Oak/845/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/845/console] > Build Jackrabbit Oak #844 failed > > > Key: OAK-6795 > URL: https://issues.apache.org/jira/browse/OAK-6795 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #844 has failed. > First failed run: [Jackrabbit Oak > #844|https://builds.apache.org/job/Jackrabbit%20Oak/844/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/844/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (OAK-6750) Lucene facets don't work with relative properties
[
https://issues.apache.org/jira/browse/OAK-6750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh resolved OAK-6750.
Resolution: Fixed
Fix Version/s: 1.7.9
Fixed on trunk at [r1811380|https://svn.apache.org/r1811380].
> Lucene facets don't work with relative properties
> -
>
> Key: OAK-6750
> URL: https://issues.apache.org/jira/browse/OAK-6750
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Labels: candidate_oak_1_4, candidate_oak_1_6
> Fix For: 1.8, 1.7.9
>
> Attachments: OAK-6750-testCase.patch
>
>
> Facets currently work only when faceted dimension is direct property of
> selected node e.g.
> {noformat}
> select \[jcr:path], \[rep:facet(jc/text)] from \[nt:base] where ...
> {noformat}
> doesn't work but
> {noformat}
> select \[jcr:path], \[rep:facet(text)] from \[nt:base] where ...
> {noformat}
> works (assume correct index defs).
> The issue is that {{ast/SelectorImpl#currentOakProperty}} check if
> {{rep:facet(jc/text)}} is a relative path or not and incorrectly 'feels' that
> it indeed is - it then proceeds ahead to traverse down a tree with elements
> \[{{rep:facet(jc}}, {{text)}}\] which makes no sense.
> /cc [~teofili], [~tmueller]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195544#comment-16195544 ] Vikas Saurabh commented on OAK-6787: Patch lgtm. bq. was thinking to use this when NodeStore is of type Clusterable I'm guessing you are saying that seg-store can do a a very fast sub-tree deletion!? If so, yes, I think that would be useful - I guess that check would happen at the wiring end though, right? > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6787-v1.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6792) rep:facet not supported in xpath
[
https://issues.apache.org/jira/browse/OAK-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh updated OAK-6792:
---
Description:
Parsing
{noformat}
//element(*, app:Asset)/(rep:facet(jcr:createdBy))
{noformat}
gives following exception.
{noformat}
Caused by: java.text.ParseException: Query:
/jcr_root//element(*, app:Asset)/rep:facet(jcr:createdBy(*))
{noformat}
/cc [~tmueller]
was:
Parsing
{noformat}
//element(*, app:Asset)[rep:facet(jcr:createdBy)]
{noformat}
gives following exception.
{noformat}
javax.jcr.query.InvalidQueryException: java.text.ParseException: Query:
//element(*, app:Asset)[rep:facet(jcr:createdBy(*))]; expected: jcr:like |
jcr:contains | jcr:score | xs:dateTime | fn:lower-case | fn:upper-case |
fn:name | rep:similar | rep:spellcheck | rep:suggest
{noformat}
/cc [~tmueller]
> rep:facet not supported in xpath
>
>
> Key: OAK-6792
> URL: https://issues.apache.org/jira/browse/OAK-6792
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Vikas Saurabh
>
> Parsing
> {noformat}
> //element(*, app:Asset)/(rep:facet(jcr:createdBy))
> {noformat}
> gives following exception.
> {noformat}
> Caused by: java.text.ParseException: Query:
> /jcr_root//element(*, app:Asset)/rep:facet(jcr:createdBy(*))
> {noformat}
> /cc [~tmueller]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6735) Lucene Index: improved cost estimation by using document count per field
[
https://issues.apache.org/jira/browse/OAK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194824#comment-16194824
]
Vikas Saurabh commented on OAK-6735:
bq. I would be in favour of supporting customer defined selectivity per
property. See also:
Quick question: do you mean that:
* we generate some statistics which end-user can
scale-up/down/completely-override according to domain knowledge (I don't feel
nice about complete over-ride though), OR
* end user has some way to force refresh stored statistics (like {{ANALYZE}}
command in links posted above), OR
* no auto stats generation. End user can give some inputs in index def to give
some hint for cost estimation though?
> Lucene Index: improved cost estimation by using document count per field
>
>
> Key: OAK-6735
> URL: https://issues.apache.org/jira/browse/OAK-6735
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: lucene, query
>Affects Versions: 1.7.4
>Reporter: Thomas Mueller
> Fix For: 1.8
>
>
> The cost estimation of the Lucene index is somewhat inaccurate because (by
> default) it just used the number of documents in the index (as of Oak 1.7.4
> by default, due to OAK-6333).
> Instead, it should use the number of documents for the given fields (the
> minimum, if there are multiple fields with restrictions).
> Plus divided by the number of restrictions (as we do now already).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6550) Make BlobTracker snaphot process smarter
[ https://issues.apache.org/jira/browse/OAK-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194749#comment-16194749 ] Amit Jain commented on OAK-6550: [~catholicon] Yes this should also be solved once this is fixed i.e. once the active deletions are synced with BlobTracker immediately. > Make BlobTracker snaphot process smarter > > > Key: OAK-6550 > URL: https://issues.apache.org/jira/browse/OAK-6550 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-plugins >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.8 > > > Currently, the BlobTracker snapshot() process is scheduled (every 12 hours by > default) which synchronizes the local additions & deletions since last change > with the datastore and consequently other cluster nodes. > The process should be slightly improved to run the sync on every removal i.e. > either through : > * Blob GC > * Active Deletion > And add some heuristic (most likely time based) to skip an upcoming sync > cycle if it makes sense. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6795) Build Jackrabbit Oak #844 failed
Hudson created OAK-6795: --- Summary: Build Jackrabbit Oak #844 failed Key: OAK-6795 URL: https://issues.apache.org/jira/browse/OAK-6795 Project: Jackrabbit Oak Issue Type: Bug Components: continuous integration Reporter: Hudson No description is provided The build Jackrabbit Oak #844 has failed. First failed run: [Jackrabbit Oak #844|https://builds.apache.org/job/Jackrabbit%20Oak/844/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/844/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6788) java.io.IOException: Backing channel 'ubuntu-1' is disconnected.
[ https://issues.apache.org/jira/browse/OAK-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194728#comment-16194728 ] Hudson commented on OAK-6788: - Previously failing build now is OK. Passed run: [Jackrabbit Oak #843|https://builds.apache.org/job/Jackrabbit%20Oak/843/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/843/console] > java.io.IOException: Backing channel 'ubuntu-1' is disconnected. > > > Key: OAK-6788 > URL: https://issues.apache.org/jira/browse/OAK-6788 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #842 has failed. > First failed run: [Jackrabbit Oak > #842|https://builds.apache.org/job/Jackrabbit%20Oak/842/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/842/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6550) Make BlobTracker snaphot process smarter
[
https://issues.apache.org/jira/browse/OAK-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194720#comment-16194720
]
Vikas Saurabh commented on OAK-6550:
Follow up to previous comment - this is the warn noticed during blob store
consistency check:
{noformat}
06.10.2017 13:41:55.392 *INFO* [sling-oak-observation-41359]
org.apache.jackrabbit.oak.plugins.blob.MarkSweepGarbageCollector Number of
blobs present in BlobStore : [410869]
06.10.2017 13:41:55.392 *INFO* [sling-oak-observation-41359]
org.apache.jackrabbit.oak.plugins.blob.MarkSweepGarbageCollector Length of blob
ids file retrieved 28896382
06.10.2017 13:41:56.810 *INFO* [sling-oak-observation-41346]
org.apache.jackrabbit.oak.plugins.blob.MarkSweepGarbageCollector Consistency
check found [136002] missing blobs
06.10.2017 13:41:56.811 *WARN* [sling-oak-observation-41346]
org.apache.jackrabbit.oak.plugins.blob.MarkSweepGarbageCollector Consistency
check failure in the the blob store : DataStore backed BlobStore
[org.apache.jackrabbit.oak.plugins.blob.datastore.OakFileDataStore], check
missing candidates in file /tmp/gcworkdir-1507296968243/gccand-1507296968243
{noformat}
> Make BlobTracker snaphot process smarter
>
>
> Key: OAK-6550
> URL: https://issues.apache.org/jira/browse/OAK-6550
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: blob-plugins
>Reporter: Amit Jain
>Assignee: Amit Jain
> Fix For: 1.8
>
>
> Currently, the BlobTracker snapshot() process is scheduled (every 12 hours by
> default) which synchronizes the local additions & deletions since last change
> with the datastore and consequently other cluster nodes.
> The process should be slightly improved to run the sync on every removal i.e.
> either through :
> * Blob GC
> * Active Deletion
> And add some heuristic (most likely time based) to skip an upcoming sync
> cycle if it makes sense.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6735) Lucene Index: improved cost estimation by using document count per field
[ https://issues.apache.org/jira/browse/OAK-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194679#comment-16194679 ] Thomas Mueller commented on OAK-6735: - I would be in favour of supporting customer defined selectivity per property. See also: * https://www.postgresql.org/docs/8.1/static/planner-stats-details.html (search for selectivity) * https://sqlite.org/fileformat2.html#stat4tab * http://www.h2database.com/html/grammar.html#analyze (see selectivity) > Lucene Index: improved cost estimation by using document count per field > > > Key: OAK-6735 > URL: https://issues.apache.org/jira/browse/OAK-6735 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene, query >Affects Versions: 1.7.4 >Reporter: Thomas Mueller > Fix For: 1.8 > > > The cost estimation of the Lucene index is somewhat inaccurate because (by > default) it just used the number of documents in the index (as of Oak 1.7.4 > by default, due to OAK-6333). > Instead, it should use the number of documents for the given fields (the > minimum, if there are multiple fields with restrictions). > Plus divided by the number of restrictions (as we do now already). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6633) Overwriting a versionable node with the copy-versions=false doesn't remove versionable properties
[
https://issues.apache.org/jira/browse/OAK-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194666#comment-16194666
]
Tomek Rękawek commented on OAK-6633:
Fixed for trunk in [r1811337|https://svn.apache.org/r1811337].
> Overwriting a versionable node with the copy-versions=false doesn't remove
> versionable properties
> -
>
> Key: OAK-6633
> URL: https://issues.apache.org/jira/browse/OAK-6633
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.8, 1.7.9
>
>
> Steps to reproduce:
> 1. Migrate a versionable node, including its version history
> ({{\-\-copy-versions=true}})
> 2. Migrate it again, excluding its version history
> ({{\-\-copy-versions=false}}
> Result:
> * the node is still versionable after the 2nd migration
> * the version history is not removed
> Expected:
> * the node shouldn't be versionable after the 2nd migration
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (OAK-6633) Overwriting a versionable node with the copy-versions=false doesn't remove versionable properties
[
https://issues.apache.org/jira/browse/OAK-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek resolved OAK-6633.
Resolution: Fixed
> Overwriting a versionable node with the copy-versions=false doesn't remove
> versionable properties
> -
>
> Key: OAK-6633
> URL: https://issues.apache.org/jira/browse/OAK-6633
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.8, 1.7.9
>
>
> Steps to reproduce:
> 1. Migrate a versionable node, including its version history
> ({{\-\-copy-versions=true}})
> 2. Migrate it again, excluding its version history
> ({{\-\-copy-versions=false}}
> Result:
> * the node is still versionable after the 2nd migration
> * the version history is not removed
> Expected:
> * the node shouldn't be versionable after the 2nd migration
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6633) Overwriting a versionable node with the copy-versions=false doesn't remove versionable properties
[
https://issues.apache.org/jira/browse/OAK-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-6633:
---
Fix Version/s: 1.7.9
> Overwriting a versionable node with the copy-versions=false doesn't remove
> versionable properties
> -
>
> Key: OAK-6633
> URL: https://issues.apache.org/jira/browse/OAK-6633
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.8, 1.7.9
>
>
> Steps to reproduce:
> 1. Migrate a versionable node, including its version history
> ({{\-\-copy-versions=true}})
> 2. Migrate it again, excluding its version history
> ({{\-\-copy-versions=false}}
> Result:
> * the node is still versionable after the 2nd migration
> * the version history is not removed
> Expected:
> * the node shouldn't be versionable after the 2nd migration
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6776) Correctly use IndexPlan.supportsPathRestrictions
[ https://issues.apache.org/jira/browse/OAK-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194639#comment-16194639 ] Thomas Mueller commented on OAK-6776: - http://svn.apache.org/r1811334 (trunk) still missing is: lucene indexes should return multiple paths, if some indexes do support path restrictions and some not > Correctly use IndexPlan.supportsPathRestrictions > > > Key: OAK-6776 > URL: https://issues.apache.org/jira/browse/OAK-6776 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.8 > > > Right now, IndexPlan.supportsPathRestrictions (introduced in OAK-6734) is > used in the query engine for some kind of mixed "rule based" and "cost based" > [query optimization|https://en.wikipedia.org/wiki/Query_optimization]. > I think the current implementation isn't correct, as (for example) a query > with multiple indexes will now use the wrong index in some cases (for example > property index, even if the cost of the Lucene index is lower). > Also, if there is a Lucene index with supportsPathRestrictions, and one > without, right now always the one with supportsPathRestrictions is used. This > is probably better right now, but once OAK-6735 is resolved, this should be > fixed as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6789) RDB: RevisionGC performance on Oracle
[
https://issues.apache.org/jira/browse/OAK-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-6789:
Attachment: OAK-6789.diff
Potential fix that swiches to simpler but batched DELETEs instead.
> RDB: RevisionGC performance on Oracle
> -
>
> Key: OAK-6789
> URL: https://issues.apache.org/jira/browse/OAK-6789
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Attachments: OAK-6789.diff
>
>
> In RevisionGC on Oracle, performance of deletes is bad. Using RevisionGCTest:
> {noformat}
> VersionGCStats{ignoredGCDueToCheckPoint=false, canceled=false,
> deletedDocGCCount=92092 (of which leaf: 92000), updateResurrectedGCCount=0,
> splitDocGCCount=1, intermediateSplitDocGCCount=0, iterationCount=2,
> timeActive=7.429 min, timeToCollectDeletedDocs=1394 ms,
> timeToCheckDeletedDocs=250.4 ms, timeToSortDocIds=374.0 us,
> timeTakenToUpdateResurrectedDocs=17.00 us, timeTakenToDeleteDeletedDocs=7.399
> min, timeTakenToCollectAndDeleteSplitDocs=97.95 ms}
> Performed RevisionGC in 7,429 min
> {noformat}
> Compared with DB2:
> {noformat}
> VersionGCStats{ignoredGCDueToCheckPoint=false, canceled=false,
> deletedDocGCCount=96096 (of which leaf: 96000), updateResurrectedGCCount=0,
> splitDocGCCount=1, intermediateSplitDocGCCount=0, iterationCount=2,
> timeActive=8.240 s, timeToCollectDeletedDocs=1780 ms,
> timeToCheckDeletedDocs=259.7 ms, timeToSortDocIds=237.0 us,
> timeTakenToUpdateResurrectedDocs=19.00 us, timeTakenToDeleteDeletedDocs=4.552
> s, timeTakenToCollectAndDeleteSplitDocs=685.4 ms}
> Performed RevisionGC in 8,243 s
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6792) rep:facet not supported in xpath
[
https://issues.apache.org/jira/browse/OAK-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194577#comment-16194577
]
Vikas Saurabh commented on OAK-6792:
Oh... yes, you're right... I wanted that to be a column in result row... not a
condition. (so, yes I wanted {{/jcr_root//element(*,
app:Asset)/(rep:facet(jcr:createdBy))}}. Let me run the query again in a bit -
I'd update the description accordingly.
> rep:facet not supported in xpath
>
>
> Key: OAK-6792
> URL: https://issues.apache.org/jira/browse/OAK-6792
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Vikas Saurabh
>
> Parsing
> {noformat}
> //element(*, app:Asset)[rep:facet(jcr:createdBy)]
> {noformat}
> gives following exception.
> {noformat}
> javax.jcr.query.InvalidQueryException: java.text.ParseException: Query:
> //element(*, app:Asset)[rep:facet(jcr:createdBy(*))]; expected: jcr:like |
> jcr:contains | jcr:score | xs:dateTime | fn:lower-case | fn:upper-case |
> fn:name | rep:similar | rep:spellcheck | rep:suggest
> {noformat}
> /cc [~tmueller]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6550) Make BlobTracker snaphot process smarter
[ https://issues.apache.org/jira/browse/OAK-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194571#comment-16194571 ] Vikas Saurabh commented on OAK-6550: [~amjain], just for heads up - one of my team member reported that blob store consistency check also feels bad about active deletion removing blobs behind its back. Btw, at this time, it is just a guess that consistency check logging warn for missing blobs is due to active deletion (but, that surely is the most likely candidate)... > Make BlobTracker snaphot process smarter > > > Key: OAK-6550 > URL: https://issues.apache.org/jira/browse/OAK-6550 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob-plugins >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.8 > > > Currently, the BlobTracker snapshot() process is scheduled (every 12 hours by > default) which synchronizes the local additions & deletions since last change > with the datastore and consequently other cluster nodes. > The process should be slightly improved to run the sync on every removal i.e. > either through : > * Blob GC > * Active Deletion > And add some heuristic (most likely time based) to skip an upcoming sync > cycle if it makes sense. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6794) oak-upgrade fails to copy nt:file node if it was previously moved
[
https://issues.apache.org/jira/browse/OAK-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yegor Kozlov updated OAK-6794:
--
Attachment: UniquenessConstraintViolatedTest.patch
failing unit test
> oak-upgrade fails to copy nt:file node if it was previously moved
> -
>
> Key: OAK-6794
> URL: https://issues.apache.org/jira/browse/OAK-6794
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: upgrade
>Reporter: Yegor Kozlov
> Attachments: UniquenessConstraintViolatedTest.patch
>
>
> I'm using crx2oak to do blue-green deployments. Delta migration on certain
> content paths is consistently failing with the folowing error:
> {code}
> 06.10.2017 12:53:02.355 ERROR c.a.g.c.e.MigrationEngine: The internal error
> occurred during migration. The migration process has been aborted and it is
> not complete!
> java.lang.RuntimeException: javax.jcr.RepositoryException: Failed to copy
> content
> at com.google.common.io.Closer.rethrow(Closer.java:149)
> at
> com.adobe.granite.crx2oak.engine.crx.CrxUpgradeEngine.migrateCrx2(CrxUpgradeEngine.java:102)
> at
> com.adobe.granite.crx2oak.engine.MigrationEngine$1.doMigration(MigrationEngine.java:53)
> at
> com.adobe.granite.crx2oak.engine.MigrationEngine.process(MigrationEngine.java:92)
> at com.adobe.granite.crx2oak.pipeline.Pipeline.run(Pipeline.java:103)
> at com.adobe.granite.crx2oak.CRX2Oak.run(CRX2Oak.java:66)
> at com.adobe.granite.crx2oak.CRX2Oak.main(CRX2Oak.java:51)
> Caused by: javax.jcr.RepositoryException: Failed to copy content
> at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:561)
> at
> com.adobe.granite.crx2oak.engine.crx.CrxUpgradeEngine.upgradeFromCrx2(CrxUpgradeEngine.java:80)
> at
> com.adobe.granite.crx2oak.engine.crx.CrxUpgradeEngine.migrateCrx2(CrxUpgradeEngine.java:100)
> ... 5 common frames omitted
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException:
> OakConstraint0030: Uniqueness constraint violated at path [/] for one of the
> property in [jcr:uuid] having value dce2a7ce-f0b7-45b0-a0a9-f46c1d2bddd4
> at
> org.apache.jackrabbit.oak.plugins.index.property.PropertyIndexEditor.checkUniquenessConstraints(PropertyIndexEditor.java:319)
> at
> org.apache.jackrabbit.oak.plugins.index.property.PropertyIndexEditor.updateIndex(PropertyIndexEditor.java:299)
> at
> org.apache.jackrabbit.oak.plugins.index.property.PropertyIndexEditor.leave(PropertyIndexEditor.java:239)
> at
> org.apache.jackrabbit.oak.spi.commit.ProgressNotificationEditor.leave(ProgressNotificationEditor.java:76)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:56)
> at
> org.apache.jackrabbit.oak.plugins.index.IndexUpdate.enter(IndexUpdate.java:169)
> at
> org.apache.jackrabbit.oak.spi.commit.CompositeEditor.enter(CompositeEditor.java:66)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:49)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:55)
> at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade$LoggingCompositeHook.processCommit(RepositoryUpgrade.java:1049)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.prepare(SegmentNodeStore.java:604)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.optimisticMerge(SegmentNodeStore.java:634)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.execute(SegmentNodeStore.java:690)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:334)
> at
> org.apache.jackrabbit.oak.spi.state.ProxyNodeStore.merge(ProxyNodeStore.java:43)
> at
> org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:557)
> ... 7 common frames omitted
> {code}
> It appears to happen when an nt:file node was moved in the source repository
> and crx2oak is trying to copy the moved node to the destination. I was able
> to isolate it in a unit test, see attached.
> The steps to reproduce are:
> # initial migration copies the nt:file node to the destination
> # the nt:file node is moved in the source repository
> # oak-upgrade copies the moved nt:file to the target with --merge-paths
> option .
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6794) oak-upgrade fails to copy nt:file node if it was previously moved
Yegor Kozlov created OAK-6794:
-
Summary: oak-upgrade fails to copy nt:file node if it was
previously moved
Key: OAK-6794
URL: https://issues.apache.org/jira/browse/OAK-6794
Project: Jackrabbit Oak
Issue Type: Bug
Components: upgrade
Reporter: Yegor Kozlov
I'm using crx2oak to do blue-green deployments. Delta migration on certain
content paths is consistently failing with the folowing error:
{code}
06.10.2017 12:53:02.355 ERROR c.a.g.c.e.MigrationEngine: The internal error
occurred during migration. The migration process has been aborted and it is not
complete!
java.lang.RuntimeException: javax.jcr.RepositoryException: Failed to copy
content
at com.google.common.io.Closer.rethrow(Closer.java:149)
at
com.adobe.granite.crx2oak.engine.crx.CrxUpgradeEngine.migrateCrx2(CrxUpgradeEngine.java:102)
at
com.adobe.granite.crx2oak.engine.MigrationEngine$1.doMigration(MigrationEngine.java:53)
at
com.adobe.granite.crx2oak.engine.MigrationEngine.process(MigrationEngine.java:92)
at com.adobe.granite.crx2oak.pipeline.Pipeline.run(Pipeline.java:103)
at com.adobe.granite.crx2oak.CRX2Oak.run(CRX2Oak.java:66)
at com.adobe.granite.crx2oak.CRX2Oak.main(CRX2Oak.java:51)
Caused by: javax.jcr.RepositoryException: Failed to copy content
at
org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:561)
at
com.adobe.granite.crx2oak.engine.crx.CrxUpgradeEngine.upgradeFromCrx2(CrxUpgradeEngine.java:80)
at
com.adobe.granite.crx2oak.engine.crx.CrxUpgradeEngine.migrateCrx2(CrxUpgradeEngine.java:100)
... 5 common frames omitted
Caused by: org.apache.jackrabbit.oak.api.CommitFailedException:
OakConstraint0030: Uniqueness constraint violated at path [/] for one of the
property in [jcr:uuid] having value dce2a7ce-f0b7-45b0-a0a9-f46c1d2bddd4
at
org.apache.jackrabbit.oak.plugins.index.property.PropertyIndexEditor.checkUniquenessConstraints(PropertyIndexEditor.java:319)
at
org.apache.jackrabbit.oak.plugins.index.property.PropertyIndexEditor.updateIndex(PropertyIndexEditor.java:299)
at
org.apache.jackrabbit.oak.plugins.index.property.PropertyIndexEditor.leave(PropertyIndexEditor.java:239)
at
org.apache.jackrabbit.oak.spi.commit.ProgressNotificationEditor.leave(ProgressNotificationEditor.java:76)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:56)
at
org.apache.jackrabbit.oak.plugins.index.IndexUpdate.enter(IndexUpdate.java:169)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.enter(CompositeEditor.java:66)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:49)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:55)
at
org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade$LoggingCompositeHook.processCommit(RepositoryUpgrade.java:1049)
at
org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.prepare(SegmentNodeStore.java:604)
at
org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.optimisticMerge(SegmentNodeStore.java:634)
at
org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.execute(SegmentNodeStore.java:690)
at
org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:334)
at
org.apache.jackrabbit.oak.spi.state.ProxyNodeStore.merge(ProxyNodeStore.java:43)
at
org.apache.jackrabbit.oak.upgrade.RepositoryUpgrade.copy(RepositoryUpgrade.java:557)
... 7 common frames omitted
{code}
It appears to happen when an nt:file node was moved in the source repository
and crx2oak is trying to copy the moved node to the destination. I was able to
isolate it in a unit test, see attached.
The steps to reproduce are:
# initial migration copies the nt:file node to the destination
# the nt:file node is moved in the source repository
# oak-upgrade copies the moved nt:file to the target with --merge-paths option
.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6793) Create documentation for MBeans related to the SegmentStore/BlobStore/FileStore
Jörg Hoh created OAK-6793: - Summary: Create documentation for MBeans related to the SegmentStore/BlobStore/FileStore Key: OAK-6793 URL: https://issues.apache.org/jira/browse/OAK-6793 Project: Jackrabbit Oak Issue Type: Documentation Components: doc Affects Versions: 1.6.0 Reporter: Jörg Hoh At the moment there is no documentation available describing the MBeans of the SegmentStore except the SegmentRevisionGC. Affected MBeans as of Oak 1.6: org.apache.jackrabbit.oak:name=SegmentNodeStore statistics, type=SegmentStoreStats org.apache.jackrabbit.oak:name=Segment node store blob garbage collection,type=BlobGarbageCollection org.apache.jackrabbit.oak:name=DataStoreBlobStore,type=BlobStoreStats org.apache.jackrabbit.oak:name=FileStore statistics,type=FileStoreStats (it might be useful to have the description of SegmentRevisionGarbageCollection on the same page as the other MBeans of the SegmentNodeStore) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6792) rep:facet not supported in xpath
[
https://issues.apache.org/jira/browse/OAK-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194543#comment-16194543
]
Thomas Mueller commented on OAK-6792:
-
Yes, currently it looks like this only works with SQL-2, according to the docs
at http://jackrabbit.apache.org/oak/docs/query/query-engine.html#Facets
But I think the syntax would need to be slightly different, as the above would
be converted to "where [rep:facet(jcr:createdBy)] is not null". You probably
(?) want something like this (see xpath syntax for
[suggestions|http://jackrabbit.apache.org/oak/docs/query/query-engine.html#Suggestions]):
{noformat}
/jcr:root//*/(rep:facet(jcr:createdBy))
{noformat}
> rep:facet not supported in xpath
>
>
> Key: OAK-6792
> URL: https://issues.apache.org/jira/browse/OAK-6792
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Vikas Saurabh
>
> Parsing
> {noformat}
> //element(*, app:Asset)[rep:facet(jcr:createdBy)]
> {noformat}
> gives following exception.
> {noformat}
> javax.jcr.query.InvalidQueryException: java.text.ParseException: Query:
> //element(*, app:Asset)[rep:facet(jcr:createdBy(*))]; expected: jcr:like |
> jcr:contains | jcr:score | xs:dateTime | fn:lower-case | fn:upper-case |
> fn:name | rep:similar | rep:spellcheck | rep:suggest
> {noformat}
> /cc [~tmueller]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6792) rep:facet not supported in xpath
Vikas Saurabh created OAK-6792:
--
Summary: rep:facet not supported in xpath
Key: OAK-6792
URL: https://issues.apache.org/jira/browse/OAK-6792
Project: Jackrabbit Oak
Issue Type: Improvement
Components: query
Reporter: Vikas Saurabh
Parsing
{noformat}
//element(*, app:Asset)[rep:facet(jcr:createdBy)]
{noformat}
gives following exception.
{noformat}
javax.jcr.query.InvalidQueryException: java.text.ParseException: Query:
//element(*, app:Asset)[rep:facet(jcr:createdBy(*))]; expected: jcr:like |
jcr:contains | jcr:score | xs:dateTime | fn:lower-case | fn:upper-case |
fn:name | rep:similar | rep:spellcheck | rep:suggest
{noformat}
/cc [~tmueller]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6791) Create documentation for Indexing related MBeans
Jörg Hoh created OAK-6791: - Summary: Create documentation for Indexing related MBeans Key: OAK-6791 URL: https://issues.apache.org/jira/browse/OAK-6791 Project: Jackrabbit Oak Issue Type: Documentation Affects Versions: 1.6.1 Reporter: Jörg Hoh At the moment there is no documentation on the information provided by the Oak MBeans or the actions which can be triggered. Instead of just having inline documentation (which sometimes is available, sometimes not) it would be much better if the documentation is provided on the Oak site in a consolidated manner. The affected MBeans for Indexing as of Oak 1.6: org.apache.jackrabbit.oak:name=IndexCopierStats,type=IndexCopierStats org.apache.jackrabbit.oak:name=async,type=IndexStats org.apache.jackrabbit.oak:name=async-reindex,type=IndexStats org.apache.jackrabbit.oak:name=fulltext-async,type=IndexStats org.apache.jackrabbit.oak:name=Lucene Index statistics, type=LuceneIndex org.apache.jackrabbit.oak:name=async,type=PropertyIndexAsyncReindex org.apache.jackrabbit.oak:name=Property Index statistics, type=PropertyIndexStats (A sample of a good documentation is the documentation of the SegmentRevisionGarbageCollection available at [1]). [1] http://jackrabbit.apache.org/oak/docs/nodestore/segment/overview.html#monitoring-via-jmx -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (OAK-6789) RDB: RevisionGC performance on Oracle
[
https://issues.apache.org/jira/browse/OAK-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194484#comment-16194484
]
Julian Reschke edited comment on OAK-6789 at 10/6/17 11:50 AM:
---
Further tests show that it's indeed the condition on MODIFIED that triggers the
behavior.
Removing it (breaking the API contract...) fixes the performance problem. So
does adding an index on MODIFIED:
{noformat}
--
| Id | Operation | Name | Rows | Bytes |
Cost (%CPU)| Time |
--
| 0 | DELETE STATEMENT | | 1 | 122 |
20 (0)| 00:00:01 |
| 1 | DELETE | NODES| | |
| |
| 2 | INLIST ITERATOR | | | |
| |
|* 3 |TABLE ACCESS BY INDEX ROWID BATCHED| NODES| 1 | 122 |
20 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | MODIFIED_IDX | 8 | |
18 (0)| 00:00:01 |
--
Predicate Information (identified by operation id):
---
3 - filter("ID"='x1' AND "MODIFIED"=1 OR "ID"='x10' AND "MODIFIED"=10 OR "ID"
='x11' AND
"MODIFIED"=11 OR "ID"='x12' AND "MODIFIED"=12 OR "ID"='x13' AND "M
ODIFIED"=13 OR "ID"='x14'
AND "MODIFIED"=14 OR "ID"='x15' AND "MODIFIED"=15 OR "ID"='x16' AN
D "MODIFIED"=16 OR
"ID"='x2' AND "MODIFIED"=2 OR "ID"='x3' AND "MODIFIED"=3 OR "ID"='
x4' AND "MODIFIED"=4 OR
"ID"='x5' AND "MODIFIED"=5 OR "ID"='x6' AND "MODIFIED"=6 OR "ID"='
x7' AND "MODIFIED"=7 OR
"ID"='x8' AND "MODIFIED"=8 OR "ID"='x9' AND "MODIFIED"=9)
4 - access("MODIFIED"=1 OR "MODIFIED"=10 OR "MODIFIED"=11 OR "MODIFIED"=12 OR
"MODIFIED"=13 OR "MODIFIED"=14 OR "MODIFIED"=15 OR "MODIFIED"=16 O
R "MODIFIED"=2 OR
"MODIFIED"=3 OR "MODIFIED"=4 OR "MODIFIED"=5 OR "MODIFIED"=6 OR "M
ODIFIED"=7 OR "MODIFIED"=8
OR "MODIFIED"=9)
{noformat}
Note that this switches step 4 to an index scan on MODIFIED_IDX rather than the
primary key index. This fixes the performance in the test case, but might be
undesirable in large repos where many rows have the same modified date.
It would be interesting to understand why Oracle prefers the MODIFIED_IDX over
the primrary key index, when present. FWWI; it would be possible to add a hint
to the query to force Oracle to use a specific index (not tried over JDBC,
through).
Finally for completeness: the plan when we have a composite index on ID and
MODIFIED:
{noformat}
| Id | Operation | Name| Rows | Bytes | Cost (%CPU)|
Time |
--
| 0 | DELETE STATEMENT | | 1 | 122 |19 (0)|
00:00:01 |
| 1 | DELETE| NODES | | ||
|
| 2 | INLIST ITERATOR | | | ||
|
|* 3 |INDEX RANGE SCAN| ID_MODIFIED_IDX | 1 | 122 |19 (0)|
00:00:01 |
--
Predicate Information (identified by operation id):
---
3 - access(("ID"='x1' AND "MODIFIED"=1 OR "ID"='x10' AND "MODIFIED"=10 OR
"ID"='x11' AND "MODIFIED"=11 OR "ID"='x12' AND "MODIFIED"=12 OR "I
D"='x13'
AND "MODIFIED"=13 OR "ID"='x14' AND "MODIFIED"=14 OR "ID"='x15' AN
D
"MODIFIED"=15 OR "ID"='x16' AND "MODIFIED"=16 OR "ID"='x2' AND "MO
DIFIED"=2
OR "ID"='x3' AND "MODIFIED"=3 OR "ID"='x4' AND "MODIFIED"=4 OR "ID
"='x5' AND
"MODIFIED"=5 OR "ID"='x6' AND "MODIFIED"=6 OR "ID"='x7' AND "MODIF
IED"=7 OR
"ID"='x8' AND "MODIFIED"=8 OR "ID"='x9' AND "MODIFIED"=9))
{noformat}
which allows to process the whole condition with a single index lookup, but
might have undesirable effects elsewhere.
was (Author: reschke):
Further tests show that it's indeed the condition on MODIFIED that triggers the
behavior.
Removing it (breaking the API contract...) fixes the performance problem. So
does adding an index on MODIFIED:
{noformat}
--
| Id | Operation | Name | Rows | Bytes |
Cost (%CPU)| Time |
[jira] [Comment Edited] (OAK-6789) RDB: RevisionGC performance on Oracle
[
https://issues.apache.org/jira/browse/OAK-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194484#comment-16194484
]
Julian Reschke edited comment on OAK-6789 at 10/6/17 11:50 AM:
---
Further tests show that it's indeed the condition on MODIFIED that triggers the
behavior.
Removing it (breaking the API contract...) fixes the performance problem. So
does adding an index on MODIFIED:
{noformat}
--
| Id | Operation | Name | Rows | Bytes |
Cost (%CPU)| Time |
--
| 0 | DELETE STATEMENT | | 1 | 122 |
20 (0)| 00:00:01 |
| 1 | DELETE | NODES| | |
| |
| 2 | INLIST ITERATOR | | | |
| |
|* 3 |TABLE ACCESS BY INDEX ROWID BATCHED| NODES| 1 | 122 |
20 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | MODIFIED_IDX | 8 | |
18 (0)| 00:00:01 |
--
Predicate Information (identified by operation id):
---
3 - filter("ID"='x1' AND "MODIFIED"=1 OR "ID"='x10' AND "MODIFIED"=10 OR "ID"
='x11' AND
"MODIFIED"=11 OR "ID"='x12' AND "MODIFIED"=12 OR "ID"='x13' AND "M
ODIFIED"=13 OR "ID"='x14'
AND "MODIFIED"=14 OR "ID"='x15' AND "MODIFIED"=15 OR "ID"='x16' AN
D "MODIFIED"=16 OR
"ID"='x2' AND "MODIFIED"=2 OR "ID"='x3' AND "MODIFIED"=3 OR "ID"='
x4' AND "MODIFIED"=4 OR
"ID"='x5' AND "MODIFIED"=5 OR "ID"='x6' AND "MODIFIED"=6 OR "ID"='
x7' AND "MODIFIED"=7 OR
"ID"='x8' AND "MODIFIED"=8 OR "ID"='x9' AND "MODIFIED"=9)
4 - access("MODIFIED"=1 OR "MODIFIED"=10 OR "MODIFIED"=11 OR "MODIFIED"=12 OR
"MODIFIED"=13 OR "MODIFIED"=14 OR "MODIFIED"=15 OR "MODIFIED"=16 O
R "MODIFIED"=2 OR
"MODIFIED"=3 OR "MODIFIED"=4 OR "MODIFIED"=5 OR "MODIFIED"=6 OR "M
ODIFIED"=7 OR "MODIFIED"=8
OR "MODIFIED"=9)
{noformat}
Note that this switches step 4 to an index scan on MODIFIED_IDX rather than the
primary key index. This fixes the performance in the test case, but might be
undesirable in large repos where many rows have the same modified date.
It would be interesting to understand why Oracle prefers the MODIFIED_IDX over
the primrary key index, when present. FWWI; it would be possible to add a hint
to the query to force Oracle to use a specific index (not tried over JDBC,
through).
Finally for completeness: the plan when we have a composite index on ID and
MODIFIED:
{noformat}
| Id | Operation | Name| Rows | Bytes | Cost (%CPU)|
Time |
--
| 0 | DELETE STATEMENT | | 1 | 122 |19 (0)|
00:00:01 |
| 1 | DELETE| NODES | | ||
|
| 2 | INLIST ITERATOR | | | ||
|
|* 3 |INDEX RANGE SCAN| ID_MODIFIED_IDX | 1 | 122 |19 (0)|
00:00:01 |
--
Predicate Information (identified by operation id):
---
3 - access(("ID"='x1' AND "MODIFIED"=1 OR "ID"='x10' AND "MODIFIED"=10 OR
"ID"='x11' AND "MODIFIED"=11 OR "ID"='x12' AND "MODIFIED"=12 OR "I
D"='x13'
AND "MODIFIED"=13 OR "ID"='x14' AND "MODIFIED"=14 OR "ID"='x15' AN
D
"MODIFIED"=15 OR "ID"='x16' AND "MODIFIED"=16 OR "ID"='x2' AND "MO
DIFIED"=2
OR "ID"='x3' AND "MODIFIED"=3 OR "ID"='x4' AND "MODIFIED"=4 OR "ID
"='x5' AND
"MODIFIED"=5 OR "ID"='x6' AND "MODIFIED"=6 OR "ID"='x7' AND "MODIF
IED"=7 OR
"ID"='x8' AND "MODIFIED"=8 OR "ID"='x9' AND "MODIFIED"=9))
{noformat}
which allows to process the whole condition with a single index lookup, but
might hav undesirable effects elsewhere.
was (Author: reschke):
Further tests show that it's indeed the condition on MODIFIED that triggers the
behavior.
Removing it (breaking the API contract...) fixes the performance problem. So
does adding an index on MODIFIED:
{noformat}
--
| Id | Operation | Name | Rows | Bytes |
Cost (%CPU)| Time |
[jira] [Commented] (OAK-6789) RDB: RevisionGC performance on Oracle
[
https://issues.apache.org/jira/browse/OAK-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194484#comment-16194484
]
Julian Reschke commented on OAK-6789:
-
Further tests show that it's indeed the condition on MODIFIED that triggers the
behavior.
Removing it (breaking the API contract...) fixes the performance problem. So
does adding an index on MODIFIED:
{noformat}
--
| Id | Operation | Name | Rows | Bytes |
Cost (%CPU)| Time |
--
| 0 | DELETE STATEMENT | | 1 | 122 |
20 (0)| 00:00:01 |
| 1 | DELETE | NODES| | |
| |
| 2 | INLIST ITERATOR | | | |
| |
|* 3 |TABLE ACCESS BY INDEX ROWID BATCHED| NODES| 1 | 122 |
20 (0)| 00:00:01 |
|* 4 | INDEX RANGE SCAN | MODIFIED_IDX | 8 | |
18 (0)| 00:00:01 |
--
Predicate Information (identified by operation id):
---
3 - filter("ID"='x1' AND "MODIFIED"=1 OR "ID"='x10' AND "MODIFIED"=10 OR "ID"
='x11' AND
"MODIFIED"=11 OR "ID"='x12' AND "MODIFIED"=12 OR "ID"='x13' AND "M
ODIFIED"=13 OR "ID"='x14'
AND "MODIFIED"=14 OR "ID"='x15' AND "MODIFIED"=15 OR "ID"='x16' AN
D "MODIFIED"=16 OR
"ID"='x2' AND "MODIFIED"=2 OR "ID"='x3' AND "MODIFIED"=3 OR "ID"='
x4' AND "MODIFIED"=4 OR
"ID"='x5' AND "MODIFIED"=5 OR "ID"='x6' AND "MODIFIED"=6 OR "ID"='
x7' AND "MODIFIED"=7 OR
"ID"='x8' AND "MODIFIED"=8 OR "ID"='x9' AND "MODIFIED"=9)
4 - access("MODIFIED"=1 OR "MODIFIED"=10 OR "MODIFIED"=11 OR "MODIFIED"=12 OR
"MODIFIED"=13 OR "MODIFIED"=14 OR "MODIFIED"=15 OR "MODIFIED"=16 O
R "MODIFIED"=2 OR
"MODIFIED"=3 OR "MODIFIED"=4 OR "MODIFIED"=5 OR "MODIFIED"=6 OR "M
ODIFIED"=7 OR "MODIFIED"=8
OR "MODIFIED"=9)
{noformat}
Note that this switches step 4 to an index scan on MODIFIED_IDX rather than the
primary key index. This fixes the performance in the test case, but might be
undesirable in large repos where many rows have the same modified date.
> RDB: RevisionGC performance on Oracle
> -
>
> Key: OAK-6789
> URL: https://issues.apache.org/jira/browse/OAK-6789
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>
> In RevisionGC on Oracle, performance of deletes is bad. Using RevisionGCTest:
> {noformat}
> VersionGCStats{ignoredGCDueToCheckPoint=false, canceled=false,
> deletedDocGCCount=92092 (of which leaf: 92000), updateResurrectedGCCount=0,
> splitDocGCCount=1, intermediateSplitDocGCCount=0, iterationCount=2,
> timeActive=7.429 min, timeToCollectDeletedDocs=1394 ms,
> timeToCheckDeletedDocs=250.4 ms, timeToSortDocIds=374.0 us,
> timeTakenToUpdateResurrectedDocs=17.00 us, timeTakenToDeleteDeletedDocs=7.399
> min, timeTakenToCollectAndDeleteSplitDocs=97.95 ms}
> Performed RevisionGC in 7,429 min
> {noformat}
> Compared with DB2:
> {noformat}
> VersionGCStats{ignoredGCDueToCheckPoint=false, canceled=false,
> deletedDocGCCount=96096 (of which leaf: 96000), updateResurrectedGCCount=0,
> splitDocGCCount=1, intermediateSplitDocGCCount=0, iterationCount=2,
> timeActive=8.240 s, timeToCollectDeletedDocs=1780 ms,
> timeToCheckDeletedDocs=259.7 ms, timeToSortDocIds=237.0 us,
> timeTakenToUpdateResurrectedDocs=19.00 us, timeTakenToDeleteDeletedDocs=4.552
> s, timeTakenToCollectAndDeleteSplitDocs=685.4 ms}
> Performed RevisionGC in 8,243 s
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6789) RDB: RevisionGC performance on Oracle
[
https://issues.apache.org/jira/browse/OAK-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194472#comment-16194472
]
Julian Reschke commented on OAK-6789:
-
Further tests show that this is caused by the way we delete in batches, and the
way Oracle handles these deletes.
The version garbage collection deletes in chunks of 450 documents, each
identified by ID and with a condition on it's modification date (to prevent
deletion of documents that have changed since the collection phase).
In {{RDBDocumentStore}} we break these down to chunks of 64 documents, and
generate a single DELETE statement like this:
{noformat}
delete from nodes where id='x1' and modified=1 or id='x2' and modified=2 ...
{noformat}
Tests show that Oracle behaves sanely as long as we do not put more than 8
conditions into a single statement. Checking the execution plan for 16
conditions:
{noformat}
---
| Id | Operation | Name | Rows | Bytes | Cost
(%CPU)| Time |
---
| 0 | DELETE STATEMENT | | 1 | 122 |
25(0)| 00:00:01 |
| 1 | DELETE | NODES | | | |
|
| 2 | INLIST ITERATOR | | | | |
|
|* 3 |TABLE ACCESS BY INDEX ROWID| NODES | 1 | 122 |
25(0)| 00:00:01 |
|* 4 | INDEX UNIQUE SCAN | SYS_C00175322 | 8 | |
19(0)| 00:00:01 |
---
Predicate Information (identified by operation id):
---
3 - filter("ID"='x1' AND "MODIFIED"=1 OR "ID"='x10' AND "MODIFIED"=10 OR "ID"
='x11'
AND "MODIFIED"=11 OR "ID"='x12' AND "MODIFIED"=12 OR "ID"='x13' AN
D "MODIFIED"=13 OR
"ID"='x14' AND "MODIFIED"=14 OR "ID"='x15' AND "MODIFIED"=15 OR "I
D"='x16' AND
"MODIFIED"=16 OR "ID"='x2' AND "MODIFIED"=2 OR "ID"='x3' AND "MODI
FIED"=3 OR "ID"='x4'
AND "MODIFIED"=4 OR "ID"='x5' AND "MODIFIED"=5 OR "ID"='x6' AND "M
ODIFIED"=6 OR
"ID"='x7' AND "MODIFIED"=7 OR "ID"='x8' AND "MODIFIED"=8 OR "ID"='
x9' AND "MODIFIED"=9)
4 - access("ID"='x1' OR "ID"='x10' OR "ID"='x11' OR "ID"='x12' OR "ID"='x13'
OR
"ID"='x14' OR "ID"='x15' OR "ID"='x16' OR "ID"='x2' OR "ID"='x3' O
R "ID"='x4' OR
"ID"='x5' OR "ID"='x6' OR "ID"='x7' OR "ID"='x8' OR "ID"='x9')
{noformat}
where "SYS_C00175322" is the primary unique index on ID.
For 8 conditions:
{noformat}
---
| Id | Operation | Name | Rows | Bytes | Cost
(%CPU)| Time |
---
| 0 | DELETE STATEMENT | | 1 | 122 |
14(0)| 00:00:01 |
| 1 | DELETE | NODES | | | |
|
| 2 | INLIST ITERATOR | | | | |
|
|* 3 |TABLE ACCESS BY INDEX ROWID| NODES | 1 | 122 |
14(0)| 00:00:01 |
|* 4 | INDEX UNIQUE SCAN | SYS_C00175322 | 4 | |
11(0)| 00:00:01 |
---
Predicate Information (identified by operation id):
---
3 - filter("ID"='x1' AND "MODIFIED"=1 OR "ID"='x2' AND "MODIFIED"=2 OR "ID"='
x3'
AND "MODIFIED"=3 OR "ID"='x4' AND "MODIFIED"=4 OR "ID"='x5' AND "M
ODIFIED"=5 OR
"ID"='x6' AND "MODIFIED"=6 OR "ID"='x7' AND "MODIFIED"=7 OR "ID"='
x8' AND "MODIFIED"=8)
4 - access("ID"='x1' OR "ID"='x2' OR "ID"='x3' OR "ID"='x4' OR "ID"='x5' OR
"ID"='x6' OR "ID"='x7' OR "ID"='x8')
{noformat}
> RDB: RevisionGC performance on Oracle
> -
>
> Key: OAK-6789
> URL: https://issues.apache.org/jira/browse/OAK-6789
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>
> In RevisionGC on Oracle, performance of deletes is bad. Using RevisionGCTest:
> {noformat}
> VersionGCStats{ignoredGCDueToCheckPoint=false, canceled=false,
> deletedDocGCCount=92092 (of which leaf: 92000), updateResurrectedGCCount=0,
> splitDocGCCount=1, intermediateSplitDocGCCount=0, iterationCount=2,
> timeActive=7.429 min, timeToCollectDeletedDocs=1394 ms,
>
[jira] [Updated] (OAK-6790) FacetResult class isn't exposed anymore
[
https://issues.apache.org/jira/browse/OAK-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh updated OAK-6790:
---
Component/s: (was: lucene)
query
> FacetResult class isn't exposed anymore
> ---
>
> Key: OAK-6790
> URL: https://issues.apache.org/jira/browse/OAK-6790
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Affects Versions: 1.7.2
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Fix For: 1.8
>
>
> OAK-3847 exposed {{FacetResult}} (rather
> {{org.apache.jackrabbit.oak.query.facet}} package) to easily consume
> extracted facet data. That exported package got removed during OAK-6304 (as
> noted in this comment \[1]).
> We need to export that class - maybe that class makes sense as part of
> {{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
> if we can simply re-enable the removed export AND that its current place in
> oak-core still makes sense or not).
> [~anchela], would you remember what was the warning that was observed without
> removing this export (disclaimer: I still feel that it belongs in
> {{oak-query-spi}}... I'm just trying to have all data available).
> \[1]:
> https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6790) FacetResult class isn't exposed anymore
[
https://issues.apache.org/jira/browse/OAK-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh updated OAK-6790:
---
Description:
OAK-3847 exposed {{FacetResult}} (rather
{{org.apache.jackrabbit.oak.query.facet}} package) to easily consume extracted
facet data. That exported package got removed during OAK-6304 (as noted in this
comment \[1]).
We need to export that class - maybe that class makes sense as part of
{{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
if we can simply re-enable the removed export AND that its current place in
oak-core still makes sense or not).
[~anchela], would you remember what was the warning that was observed without
removing this export (disclaimer: I still feel that it belongs in
{{oak-query-spi}}... I'm just trying to have all data available).
\[1]:
https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
was:
OAK-3847 exposed {{FacetResult}} to easily consume extracted facet data. That
exported package got removed during OAK-6304 (as noted in this comment \[1]).
We need to export that class - maybe that class makes sense as part of
{{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
if we can simply re-enable the removed export AND that its current place in
oak-core still makes sense or not).
[~anchela], would you remember what was the warning that was observed without
removing this export (disclaimer: I still feel that it belongs in
{{oak-query-spi}}... I'm just trying to have all data available).
\[1]:
https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
> FacetResult class isn't exposed anymore
> ---
>
> Key: OAK-6790
> URL: https://issues.apache.org/jira/browse/OAK-6790
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Affects Versions: 1.7.2
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
> Fix For: 1.8
>
>
> OAK-3847 exposed {{FacetResult}} (rather
> {{org.apache.jackrabbit.oak.query.facet}} package) to easily consume
> extracted facet data. That exported package got removed during OAK-6304 (as
> noted in this comment \[1]).
> We need to export that class - maybe that class makes sense as part of
> {{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
> if we can simply re-enable the removed export AND that its current place in
> oak-core still makes sense or not).
> [~anchela], would you remember what was the warning that was observed without
> removing this export (disclaimer: I still feel that it belongs in
> {{oak-query-spi}}... I'm just trying to have all data available).
> \[1]:
> https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6790) FacetResult class isn't exposed anymore
Vikas Saurabh created OAK-6790:
--
Summary: FacetResult class isn't exposed anymore
Key: OAK-6790
URL: https://issues.apache.org/jira/browse/OAK-6790
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Affects Versions: 1.7.2
Reporter: Vikas Saurabh
Assignee: Vikas Saurabh
Fix For: 1.8
OAK-3847 exposed {{FacetResult}} to easily consume extracted facet data. That
exported package got removed during OAK-6304 (as noted in this comment \[1]).
We need to export that class - maybe that class makes sense as part of
{{oak-query-spi}} module - that still needs to be checked as assessed btw (ie
if we can simply re-enable the removed export AND that its current place in
oak-core still makes sense or not).
[~anchela], would you remember what was the warning that was observed without
removing this export (disclaimer: I still feel that it belongs in
{{oak-query-spi}}... I'm just trying to have all data available).
\[1]:
https://issues.apache.org/jira/browse/OAK-6304?focusedCommentId=16042721=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042721
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (OAK-6789) RDB: RevisionGC performance on Oracle
Julian Reschke created OAK-6789:
---
Summary: RDB: RevisionGC performance on Oracle
Key: OAK-6789
URL: https://issues.apache.org/jira/browse/OAK-6789
Project: Jackrabbit Oak
Issue Type: Bug
Components: rdbmk
Reporter: Julian Reschke
Assignee: Julian Reschke
In RevisionGC on Oracle, performance of deletes is bad. Using RevisionGCTest:
{noformat}
VersionGCStats{ignoredGCDueToCheckPoint=false, canceled=false,
deletedDocGCCount=92092 (of which leaf: 92000), updateResurrectedGCCount=0,
splitDocGCCount=1, intermediateSplitDocGCCount=0, iterationCount=2,
timeActive=7.429 min, timeToCollectDeletedDocs=1394 ms,
timeToCheckDeletedDocs=250.4 ms, timeToSortDocIds=374.0 us,
timeTakenToUpdateResurrectedDocs=17.00 us, timeTakenToDeleteDeletedDocs=7.399
min, timeTakenToCollectAndDeleteSplitDocs=97.95 ms}
Performed RevisionGC in 7,429 min
{noformat}
Compared with DB2:
{noformat}
VersionGCStats{ignoredGCDueToCheckPoint=false, canceled=false,
deletedDocGCCount=96096 (of which leaf: 96000), updateResurrectedGCCount=0,
splitDocGCCount=1, intermediateSplitDocGCCount=0, iterationCount=2,
timeActive=8.240 s, timeToCollectDeletedDocs=1780 ms,
timeToCheckDeletedDocs=259.7 ms, timeToSortDocIds=237.0 us,
timeTakenToUpdateResurrectedDocs=19.00 us, timeTakenToDeleteDeletedDocs=4.552
s, timeTakenToCollectAndDeleteSplitDocs=685.4 ms}
Performed RevisionGC in 8,243 s
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6575) Provide a secure external URL to a DataStore binary.
[
https://issues.apache.org/jira/browse/OAK-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194443#comment-16194443
]
Ian Boston edited comment on OAK-6575 at 10/6/17 10:50 AM:
---
At a bundle level yes, it is incompatable.
1.6 oak-core vs 1.8-SNAPSHOT oak-api
But not at a package level, provided the package versions are the same, which
they are.
Since OSGi at build and runtime only looks at the package versions that are
provided, you can use any bundle that contains the correct package
version. Sling Servlets Get bundle in the patch has oak-core 1.6 using
org.apache.jackrabbit.oak.api.conversion v 1.0.0. So its MANIFEST.MF will
import org.apache.jackrabbit.oak.api.conversion=[1.0.0,2.0.0).
oak-api/1.8-SNAPSHOT also provides that package so Sling Servlets GET released
using oak-core 1.6 will also work in a OSGi container satisfied by
oak-api/1.8-SNAPSHOT
Obviously if Sling Servlets GET depends on a behaviour not expressed in the API
or binds to some other service or package only available in Oak core 1.6, then
it wont work. but for this patch, and the ones in Sling it uses the api and
a common package version.
I hope that makes sense.
was (Author: ianeboston):
At a bundle level yes.
1.6 oak-core vs 1.8-SNAPSHOT oak-api
But not at a package level, provided the package versions are the same, which
they are.
Since OSGi at build and runtime only looks at the package versions that are
provided, you can use any bundle that contains the correct package
version. Sling Servlets Get bundle in the patch has oak-core 1.6 using
org.apache.jackrabbit.oak.api.conversion v 1.0.0. So its MANIFEST.MF will
import org.apache.jackrabbit.oak.api.conversion=[1.0.0,2.0.0).
oak-api/1.8-SNAPSHOT also provides that package so Sling Servlets GET released
using oak-core 1.6 will also work in a OSGi container satisfied by
oak-api/1.8-SNAPSHOT
Obviously if Sling Servlets GET depends on a behaviour not expressed in the API
or binds to some other service or package only available in Oak core 1.6, then
it wont work. but for this patch, and the ones in Sling it uses the api and
a common package version.
I hope that makes sense.
> Provide a secure external URL to a DataStore binary.
>
>
> Key: OAK-6575
> URL: https://issues.apache.org/jira/browse/OAK-6575
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: blob, core, jcr, security
>Reporter: Ian Boston
>Assignee: Amit Jain
> Fix For: 1.8, 1.7.9
>
>
> Where the DataStore is a DataStore that may be accessed over an independent
> API it would be advantageous for Oak to provide a secure URL to allow direct,
> read only access to the current immutable instance of that binary. The term
> "secure" needs to be defined, but typically it would a URL that is valid for
> a appropriately short length of time to ensure that the risk of the URL being
> used by a user that it was not intended for, is minimised. It should also
> ensure that anyone in possession of the URL could not use the information in
> the url to create a valid URL or a valid URL to a different binary.
> One example of such a URL might be a AWS Signed URL as used by AWS CloudFront
> to access private content. The signed url being signed by a private key known
> only to the Oak instance and the the CloudFront or S3 instance. The signed
> url having a significantly low ttl so that a redirect by the same client
> would work.
> Oak should only emit these URLs to sessions that could otherwise read the
> binary directly from Oak, and Oak should be in complete control of the nature
> of the url and the security mechanisms applied to the URL.
> The viability of the approach has been investigated showing that given a JCR
> Binary it is possible to get the Oak Blob Content Identifier using
> ValueImpl.getBlob((Value)jcrBinary).getContentIentifier() and form there,
> knowing the way in which the DataStore implementation transforms that into a
> pointer into the datastore implementation form a URL to be made secure.
> To achieve the above, internal implementation details specific to the Oak
> DataStore implementation are required, hence this request to implement as a
> part of Oak rather than to reverse engineer in some external project.
> Since API changes are often significant using the Sling AdapaterFactory
> approach would allow a ServletFilter to selectively use the URL in a
> redirect, avoiding any new API methods to existing Oak APIs. A new interface
> might be required, in the example below that interface is SignedBinaryURL.
> {code}
> public void doFilter(ServletRequest servletRequest, ServletResponse
> servletResponse, FilterChain filterChain) throws IOException,
> ServletException {
> if (
[jira] [Comment Edited] (OAK-6575) Provide a secure external URL to a DataStore binary.
[
https://issues.apache.org/jira/browse/OAK-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194443#comment-16194443
]
Ian Boston edited comment on OAK-6575 at 10/6/17 10:50 AM:
---
At a bundle level yes, it is incompatible.
1.6 oak-core vs 1.8-SNAPSHOT oak-api
But not at a package level, provided the package versions are the same, which
they are.
Since OSGi at build and runtime only looks at the package versions that are
provided, you can use any bundle that contains the correct package
version. Sling Servlets Get bundle in the patch has oak-core 1.6 using
org.apache.jackrabbit.oak.api.conversion v 1.0.0. So its MANIFEST.MF will
import org.apache.jackrabbit.oak.api.conversion=[1.0.0,2.0.0).
oak-api/1.8-SNAPSHOT also provides that package so Sling Servlets GET released
using oak-core 1.6 will also work in a OSGi container satisfied by
oak-api/1.8-SNAPSHOT
Obviously if Sling Servlets GET depends on a behaviour not expressed in the API
or binds to some other service or package only available in Oak core 1.6, then
it wont work. but for this patch, and the ones in Sling it uses the api and
a common package version.
I hope that makes sense.
was (Author: ianeboston):
At a bundle level yes, it is incompatable.
1.6 oak-core vs 1.8-SNAPSHOT oak-api
But not at a package level, provided the package versions are the same, which
they are.
Since OSGi at build and runtime only looks at the package versions that are
provided, you can use any bundle that contains the correct package
version. Sling Servlets Get bundle in the patch has oak-core 1.6 using
org.apache.jackrabbit.oak.api.conversion v 1.0.0. So its MANIFEST.MF will
import org.apache.jackrabbit.oak.api.conversion=[1.0.0,2.0.0).
oak-api/1.8-SNAPSHOT also provides that package so Sling Servlets GET released
using oak-core 1.6 will also work in a OSGi container satisfied by
oak-api/1.8-SNAPSHOT
Obviously if Sling Servlets GET depends on a behaviour not expressed in the API
or binds to some other service or package only available in Oak core 1.6, then
it wont work. but for this patch, and the ones in Sling it uses the api and
a common package version.
I hope that makes sense.
> Provide a secure external URL to a DataStore binary.
>
>
> Key: OAK-6575
> URL: https://issues.apache.org/jira/browse/OAK-6575
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: blob, core, jcr, security
>Reporter: Ian Boston
>Assignee: Amit Jain
> Fix For: 1.8, 1.7.9
>
>
> Where the DataStore is a DataStore that may be accessed over an independent
> API it would be advantageous for Oak to provide a secure URL to allow direct,
> read only access to the current immutable instance of that binary. The term
> "secure" needs to be defined, but typically it would a URL that is valid for
> a appropriately short length of time to ensure that the risk of the URL being
> used by a user that it was not intended for, is minimised. It should also
> ensure that anyone in possession of the URL could not use the information in
> the url to create a valid URL or a valid URL to a different binary.
> One example of such a URL might be a AWS Signed URL as used by AWS CloudFront
> to access private content. The signed url being signed by a private key known
> only to the Oak instance and the the CloudFront or S3 instance. The signed
> url having a significantly low ttl so that a redirect by the same client
> would work.
> Oak should only emit these URLs to sessions that could otherwise read the
> binary directly from Oak, and Oak should be in complete control of the nature
> of the url and the security mechanisms applied to the URL.
> The viability of the approach has been investigated showing that given a JCR
> Binary it is possible to get the Oak Blob Content Identifier using
> ValueImpl.getBlob((Value)jcrBinary).getContentIentifier() and form there,
> knowing the way in which the DataStore implementation transforms that into a
> pointer into the datastore implementation form a URL to be made secure.
> To achieve the above, internal implementation details specific to the Oak
> DataStore implementation are required, hence this request to implement as a
> part of Oak rather than to reverse engineer in some external project.
> Since API changes are often significant using the Sling AdapaterFactory
> approach would allow a ServletFilter to selectively use the URL in a
> redirect, avoiding any new API methods to existing Oak APIs. A new interface
> might be required, in the example below that interface is SignedBinaryURL.
> {code}
> public void doFilter(ServletRequest servletRequest, ServletResponse
> servletResponse, FilterChain filterChain) throws IOException,
> ServletException {
>
[jira] [Commented] (OAK-6575) Provide a secure external URL to a DataStore binary.
[
https://issues.apache.org/jira/browse/OAK-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194443#comment-16194443
]
Ian Boston commented on OAK-6575:
-
At a bundle level yes.
1.6 oak-core vs 1.8-SNAPSHOT oak-api
But not at a package level, provided the package versions are the same, which
they are.
Since OSGi at build and runtime only looks at the package versions that are
provided, you can use any bundle that contains the correct package
version. Sling Servlets Get bundle in the patch has oak-core 1.6 using
org.apache.jackrabbit.oak.api.conversion v 1.0.0. So its MANIFEST.MF will
import org.apache.jackrabbit.oak.api.conversion=[1.0.0,2.0.0).
oak-api/1.8-SNAPSHOT also provides that package so Sling Servlets GET released
using oak-core 1.6 will also work in a OSGi container satisfied by
oak-api/1.8-SNAPSHOT
Obviously if Sling Servlets GET depends on a behaviour not expressed in the API
or binds to some other service or package only available in Oak core 1.6, then
it wont work. but for this patch, and the ones in Sling it uses the api and
a common package version.
I hope that makes sense.
> Provide a secure external URL to a DataStore binary.
>
>
> Key: OAK-6575
> URL: https://issues.apache.org/jira/browse/OAK-6575
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: blob, core, jcr, security
>Reporter: Ian Boston
>Assignee: Amit Jain
> Fix For: 1.8, 1.7.9
>
>
> Where the DataStore is a DataStore that may be accessed over an independent
> API it would be advantageous for Oak to provide a secure URL to allow direct,
> read only access to the current immutable instance of that binary. The term
> "secure" needs to be defined, but typically it would a URL that is valid for
> a appropriately short length of time to ensure that the risk of the URL being
> used by a user that it was not intended for, is minimised. It should also
> ensure that anyone in possession of the URL could not use the information in
> the url to create a valid URL or a valid URL to a different binary.
> One example of such a URL might be a AWS Signed URL as used by AWS CloudFront
> to access private content. The signed url being signed by a private key known
> only to the Oak instance and the the CloudFront or S3 instance. The signed
> url having a significantly low ttl so that a redirect by the same client
> would work.
> Oak should only emit these URLs to sessions that could otherwise read the
> binary directly from Oak, and Oak should be in complete control of the nature
> of the url and the security mechanisms applied to the URL.
> The viability of the approach has been investigated showing that given a JCR
> Binary it is possible to get the Oak Blob Content Identifier using
> ValueImpl.getBlob((Value)jcrBinary).getContentIentifier() and form there,
> knowing the way in which the DataStore implementation transforms that into a
> pointer into the datastore implementation form a URL to be made secure.
> To achieve the above, internal implementation details specific to the Oak
> DataStore implementation are required, hence this request to implement as a
> part of Oak rather than to reverse engineer in some external project.
> Since API changes are often significant using the Sling AdapaterFactory
> approach would allow a ServletFilter to selectively use the URL in a
> redirect, avoiding any new API methods to existing Oak APIs. A new interface
> might be required, in the example below that interface is SignedBinaryURL.
> {code}
> public void doFilter(ServletRequest servletRequest, ServletResponse
> servletResponse, FilterChain filterChain) throws IOException,
> ServletException {
> if ( servletRequest instanceof SlingHttpServletRequest &&
> servletResponse instanceof SlingHttpServletResponse) {
> if ("GET".equals(((SlingHttpServletRequest)
> servletRequest).getMethod())){
> Resource resource = ((SlingHttpServletRequest)
> servletRequest).getResource();
> SignedBinaryURL url = resource.adaptTo(SignedBinaryURL.class);
> if (url != null) {
> ((SlingHttpServletResponse)
> servletResponse).sendRedirect(url.getURL());
> return;
> }
> }
> }
> filterChain.doFilter(servletRequest, servletResponse);
> }
> {code}
> If the AdapterFactory to go from Binary to SingedBinaryURL is not present
> then url will always be null, and no-op. If it is present, and Oak decides no
> URL is appropriate, then no-op.
> Only if the Oak DS implementation being used supports the external URL and
> Oak decides it is appropriate, will a url be available and a redirect
> performed.
> I have used AWS S3 URLs as an example, however
[jira] [Commented] (OAK-6575) Provide a secure external URL to a DataStore binary.
[
https://issues.apache.org/jira/browse/OAK-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194428#comment-16194428
]
Julian Reschke commented on OAK-6575:
-
bq. The API in 1.6 is part of oak-core, in 1.8-SNAPSHOT it is separate.
Really? Doesn't that essentially for the API between 1.6 and 1.8, and makes
1.8 incompatible with 1.6?
> Provide a secure external URL to a DataStore binary.
>
>
> Key: OAK-6575
> URL: https://issues.apache.org/jira/browse/OAK-6575
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: blob, core, jcr, security
>Reporter: Ian Boston
>Assignee: Amit Jain
> Fix For: 1.8, 1.7.9
>
>
> Where the DataStore is a DataStore that may be accessed over an independent
> API it would be advantageous for Oak to provide a secure URL to allow direct,
> read only access to the current immutable instance of that binary. The term
> "secure" needs to be defined, but typically it would a URL that is valid for
> a appropriately short length of time to ensure that the risk of the URL being
> used by a user that it was not intended for, is minimised. It should also
> ensure that anyone in possession of the URL could not use the information in
> the url to create a valid URL or a valid URL to a different binary.
> One example of such a URL might be a AWS Signed URL as used by AWS CloudFront
> to access private content. The signed url being signed by a private key known
> only to the Oak instance and the the CloudFront or S3 instance. The signed
> url having a significantly low ttl so that a redirect by the same client
> would work.
> Oak should only emit these URLs to sessions that could otherwise read the
> binary directly from Oak, and Oak should be in complete control of the nature
> of the url and the security mechanisms applied to the URL.
> The viability of the approach has been investigated showing that given a JCR
> Binary it is possible to get the Oak Blob Content Identifier using
> ValueImpl.getBlob((Value)jcrBinary).getContentIentifier() and form there,
> knowing the way in which the DataStore implementation transforms that into a
> pointer into the datastore implementation form a URL to be made secure.
> To achieve the above, internal implementation details specific to the Oak
> DataStore implementation are required, hence this request to implement as a
> part of Oak rather than to reverse engineer in some external project.
> Since API changes are often significant using the Sling AdapaterFactory
> approach would allow a ServletFilter to selectively use the URL in a
> redirect, avoiding any new API methods to existing Oak APIs. A new interface
> might be required, in the example below that interface is SignedBinaryURL.
> {code}
> public void doFilter(ServletRequest servletRequest, ServletResponse
> servletResponse, FilterChain filterChain) throws IOException,
> ServletException {
> if ( servletRequest instanceof SlingHttpServletRequest &&
> servletResponse instanceof SlingHttpServletResponse) {
> if ("GET".equals(((SlingHttpServletRequest)
> servletRequest).getMethod())){
> Resource resource = ((SlingHttpServletRequest)
> servletRequest).getResource();
> SignedBinaryURL url = resource.adaptTo(SignedBinaryURL.class);
> if (url != null) {
> ((SlingHttpServletResponse)
> servletResponse).sendRedirect(url.getURL());
> return;
> }
> }
> }
> filterChain.doFilter(servletRequest, servletResponse);
> }
> {code}
> If the AdapterFactory to go from Binary to SingedBinaryURL is not present
> then url will always be null, and no-op. If it is present, and Oak decides no
> URL is appropriate, then no-op.
> Only if the Oak DS implementation being used supports the external URL and
> Oak decides it is appropriate, will a url be available and a redirect
> performed.
> I have used AWS S3 URLs as an example, however the approach should be
> applicable (and pluggable) to most REST based APIs to private binary content.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6575) Provide a secure external URL to a DataStore binary.
[
https://issues.apache.org/jira/browse/OAK-6575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194387#comment-16194387
]
Ian Boston commented on OAK-6575:
-
Doc: Anyone deploying will find it hard to do so without the instructions in
the README.md file
https://gist.github.com/ieb/f9e044e4033b8f810238fe9a27eeaa78#file-readme-md
Could the 1.6 pull request also be merged please ?
Reasoning:
1.6: Sling will remain blocked until there is 1.6 release. Patch was backported
to unblock Sling and allow a release of the components there which require the
new API package.
The API in 1.6 is part of oak-core, in 1.8-SNAPSHOT it is separate.
IIUC Sling won't bind to an unstable (1.7.x) release of Oak so without a 1.6
release will remain blocked until there is a 1.8.0 release of Oak, which will
all continue to block downstream work.
> Provide a secure external URL to a DataStore binary.
>
>
> Key: OAK-6575
> URL: https://issues.apache.org/jira/browse/OAK-6575
> Project: Jackrabbit Oak
> Issue Type: New Feature
> Components: blob, core, jcr, security
>Reporter: Ian Boston
>Assignee: Amit Jain
> Fix For: 1.8, 1.7.9
>
>
> Where the DataStore is a DataStore that may be accessed over an independent
> API it would be advantageous for Oak to provide a secure URL to allow direct,
> read only access to the current immutable instance of that binary. The term
> "secure" needs to be defined, but typically it would a URL that is valid for
> a appropriately short length of time to ensure that the risk of the URL being
> used by a user that it was not intended for, is minimised. It should also
> ensure that anyone in possession of the URL could not use the information in
> the url to create a valid URL or a valid URL to a different binary.
> One example of such a URL might be a AWS Signed URL as used by AWS CloudFront
> to access private content. The signed url being signed by a private key known
> only to the Oak instance and the the CloudFront or S3 instance. The signed
> url having a significantly low ttl so that a redirect by the same client
> would work.
> Oak should only emit these URLs to sessions that could otherwise read the
> binary directly from Oak, and Oak should be in complete control of the nature
> of the url and the security mechanisms applied to the URL.
> The viability of the approach has been investigated showing that given a JCR
> Binary it is possible to get the Oak Blob Content Identifier using
> ValueImpl.getBlob((Value)jcrBinary).getContentIentifier() and form there,
> knowing the way in which the DataStore implementation transforms that into a
> pointer into the datastore implementation form a URL to be made secure.
> To achieve the above, internal implementation details specific to the Oak
> DataStore implementation are required, hence this request to implement as a
> part of Oak rather than to reverse engineer in some external project.
> Since API changes are often significant using the Sling AdapaterFactory
> approach would allow a ServletFilter to selectively use the URL in a
> redirect, avoiding any new API methods to existing Oak APIs. A new interface
> might be required, in the example below that interface is SignedBinaryURL.
> {code}
> public void doFilter(ServletRequest servletRequest, ServletResponse
> servletResponse, FilterChain filterChain) throws IOException,
> ServletException {
> if ( servletRequest instanceof SlingHttpServletRequest &&
> servletResponse instanceof SlingHttpServletResponse) {
> if ("GET".equals(((SlingHttpServletRequest)
> servletRequest).getMethod())){
> Resource resource = ((SlingHttpServletRequest)
> servletRequest).getResource();
> SignedBinaryURL url = resource.adaptTo(SignedBinaryURL.class);
> if (url != null) {
> ((SlingHttpServletResponse)
> servletResponse).sendRedirect(url.getURL());
> return;
> }
> }
> }
> filterChain.doFilter(servletRequest, servletResponse);
> }
> {code}
> If the AdapterFactory to go from Binary to SingedBinaryURL is not present
> then url will always be null, and no-op. If it is present, and Oak decides no
> URL is appropriate, then no-op.
> Only if the Oak DS implementation being used supports the external URL and
> Oak decides it is appropriate, will a url be available and a redirect
> performed.
> I have used AWS S3 URLs as an example, however the approach should be
> applicable (and pluggable) to most REST based APIs to private binary content.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6741) Switch to official OSGi component and metatype annotations
[
https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194320#comment-16194320
]
Julian Sedding commented on OAK-6741:
-
[~stillalex] I agree that the OSGi testing situation is currently still lacking
good support.
PaxExam seems too heavy, slow and is poorly documented IMHO. Felix Connect,
which is used in {{oak-pojosr}} seems like a promising building block, but
lacks a simple testing facade. At last week's Sling Hackathon, I suggested to
use Felix Connect under the hood for Sling's {{osgi-mock}}. If that can be done
in a backwards compatible way, that might help. However, I have a feeling that
it would still not be quite enough.
Without an investment in tooling, I unfortunately don't see how we can clean up
the OSGi test situation any time soon.
Point taken and agreed about the risks. Just for the record, I didn't take them
lightly. I tried to reduce them as much as I could and I am sorry that a few
issues seem to have slipped through.
> Switch to official OSGi component and metatype annotations
> --
>
> Key: OAK-6741
> URL: https://issues.apache.org/jira/browse/OAK-6741
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Robert Munteanu
> Fix For: 1.8, 1.7.9
>
> Attachments: OAK-6741-proposed-changes-chetans-feedback.patch,
> osgi-metadata-1.7.8.json, osgi-metadata-trunk.json
>
>
> We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi
> R6 annotations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6788) java.io.IOException: Backing channel 'ubuntu-1' is disconnected.
[
https://issues.apache.org/jira/browse/OAK-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194313#comment-16194313
]
Alex Deparvu commented on OAK-6788:
---
{noformat}
java.io.IOException: Backing channel 'ubuntu-1' is disconnected.
at
hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:192)
at
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:257)
at com.sun.proxy.$Proxy140.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1138)
at hudson.maven.ProcessCache$MavenProcess.call(ProcessCache.java:166)
at
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:879)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:496)
at hudson.model.Run.execute(Run.java:1737)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:419)
Caused by: java.io.IOException: Unexpected termination of the channel
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:73)
Caused by: java.io.EOFException
at
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638)
at
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113)
at
java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
at java.io.ObjectInputStream.(ObjectInputStream.java:349)
at
hudson.remoting.ObjectInputStreamEx.(ObjectInputStreamEx.java:48)
at
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:59)
{noformat}
> java.io.IOException: Backing channel 'ubuntu-1' is disconnected.
>
>
> Key: OAK-6788
> URL: https://issues.apache.org/jira/browse/OAK-6788
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: continuous integration
>Reporter: Hudson
>
> No description is provided
> The build Jackrabbit Oak #842 has failed.
> First failed run: [Jackrabbit Oak
> #842|https://builds.apache.org/job/Jackrabbit%20Oak/842/] [console
> log|https://builds.apache.org/job/Jackrabbit%20Oak/842/console]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6788) java.io.IOException: Backing channel 'ubuntu-1' is disconnected.
[ https://issues.apache.org/jira/browse/OAK-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Deparvu updated OAK-6788: -- Summary: java.io.IOException: Backing channel 'ubuntu-1' is disconnected. (was: Build Jackrabbit Oak #842 failed) > java.io.IOException: Backing channel 'ubuntu-1' is disconnected. > > > Key: OAK-6788 > URL: https://issues.apache.org/jira/browse/OAK-6788 > Project: Jackrabbit Oak > Issue Type: Bug > Components: continuous integration >Reporter: Hudson > > No description is provided > The build Jackrabbit Oak #842 has failed. > First failed run: [Jackrabbit Oak > #842|https://builds.apache.org/job/Jackrabbit%20Oak/842/] [console > log|https://builds.apache.org/job/Jackrabbit%20Oak/842/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6741) Switch to official OSGi component and metatype annotations
[ https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194307#comment-16194307 ] Alex Deparvu commented on OAK-6741: --- fair enough, I got a little excited as well. My main issue is generic lack of proper support for testing in an OSGi env. I don't trust the mocks to do the right thing, and can't really keep up with what the latest is in terms of supported features. It's simply too convoluted to write tests for these bits. For me cleaning up the OSGi test situation would have been a prerequisite for this work, together with increasing code coverage. No test coverage means you are taking some interesting risks while refactoring, and I'm not convinced that's a good idea. > Switch to official OSGi component and metatype annotations > -- > > Key: OAK-6741 > URL: https://issues.apache.org/jira/browse/OAK-6741 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Robert Munteanu > Fix For: 1.8, 1.7.9 > > Attachments: OAK-6741-proposed-changes-chetans-feedback.patch, > osgi-metadata-1.7.8.json, osgi-metadata-trunk.json > > > We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi > R6 annotations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (OAK-6741) Switch to official OSGi component and metatype annotations
[ https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Sedding updated OAK-6741: Attachment: OAK-6741-proposed-changes-chetans-feedback.patch Attached is a tentative patch to address [~chetanm]'s feedback. > Switch to official OSGi component and metatype annotations > -- > > Key: OAK-6741 > URL: https://issues.apache.org/jira/browse/OAK-6741 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Robert Munteanu > Fix For: 1.8, 1.7.9 > > Attachments: OAK-6741-proposed-changes-chetans-feedback.patch, > osgi-metadata-1.7.8.json, osgi-metadata-trunk.json > > > We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi > R6 annotations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6741) Switch to official OSGi component and metatype annotations
[
https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194303#comment-16194303
]
Julian Sedding commented on OAK-6741:
-
Thank you [~chetanm] for the list of differences. I'll try to address them
below.
bq. No 'Designate' found in
OSGI-INF/metatype/org.apache.jackrabbit.oak.security.authorization.AuthorizationConfigurationImpl$Configuration.xml
- Looks like AuthorizationConfigurationImpl does not have designate specified
Agreed, I think the {{@Designate}} annotation is required.
bq. COWNodeStoreService is having the "role" property missing. Which is required
You may be right. This component is confusing, however. Before my changes, it
did not have {{metatype = true}} set, yet it declares a property with (metatype
specific) {{label}} and {{description}}. Furthermore, it requires a
configuration before it becomes active.
This leads to a behaviour, where no configuration form is displayed in the web
console. But if you supply a configuration (e.g. via Slings' OSGi installer)
you _should_ specify the {{role}} property. If this is done, the service is
registered with such a property and it can be used for filtering.
So I believe that the component after my changes behaves as before. However, we
should clarify the intent, and possibly add a configuration annotation object
that is referenced in the {{@Designate}} annotation.
WDYT?
{quote}
DocumentNodeStoreService - Missing props
updateLimit
repository.home
role
{quote}
I think that's just a difference in processing. Possibly semantically
equivalent. The properties are all defined in the configuration annotation
object, but have no default values (as before).
bq. SecondaryStoreCacheService - immediate missing
This is just a component (does not expose a service). In that case {{immediate
= true}} is the default. Thereofore, I think it's fine.
bq. MountInfoProviderService - supportFragment missing (may be it was changed
so valid)
I didn't touch the {{oak-store-composite}} bundle with these changes.
{quote}
AuthenticationConfigurationImpl
org.apache.jackrabbit.oak.authentication.appName missing
org.apache.jackrabbit.oak.authentication.configSpiName missing
{quote}
We _may_ have lost {{appName}} if no configuration is present.
{{configSpiName}} has no default, so it should be fine if it's missing (it
should be present in the metatype).
bq. DefaultAuthorizableActionProvider - label and value swapped
Yes, this is wrong. It seems that the old annotations expected it the wrong way
around.
> Switch to official OSGi component and metatype annotations
> --
>
> Key: OAK-6741
> URL: https://issues.apache.org/jira/browse/OAK-6741
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Robert Munteanu
> Fix For: 1.8, 1.7.9
>
> Attachments: osgi-metadata-1.7.8.json, osgi-metadata-trunk.json
>
>
> We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi
> R6 annotations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (OAK-6741) Switch to official OSGi component and metatype annotations
[
https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194264#comment-16194264
]
Julian Sedding edited comment on OAK-6741 at 10/6/17 8:00 AM:
--
bq. Julian Sedding you might want to check your facts before pointing the
finger. we already have a test for this [0]. why this test is not failing with
the changed annotation, is a different question (it's probably the test
framework that does not respect the wiring, but I'm no OSGi expert).
_edit_: removed less constructive comment, replaced with hopefully more
constructive one
The test uses Sling's osgi-mocks via {{OsgiContext}}. This is (currently) not
backed by a true OSGi implementation and re-implements certain features like
Declarative Services and MetaType. Sometimes this can lead to problems, which
usually manifest in lacking features. However, in this case, I think that the
test should have been adjusted to bind multiple {{CredentialsSupport}} services
when OAK-6753 was done.
was (Author: jsedding):
bq. Julian Sedding you might want to check your facts before pointing the
finger. we already have a test for this [0]. why this test is not failing with
the changed annotation, is a different question (it's probably the test
framework that does not respect the wiring, but I'm no OSGi expert).
With all due respect [~stillalex], I think your quoting me out of context. As
for the facts, you say yourself that the test does not fail. So we don't have a
test that covers the scenario imlemented in OAK-6753. This is evident by the
fact that the test was not adjusted when OAK-6753 was implemented.
Anyway, we're splitting hairs. If you would like me to revert the patch, please
tell me so.
> Switch to official OSGi component and metatype annotations
> --
>
> Key: OAK-6741
> URL: https://issues.apache.org/jira/browse/OAK-6741
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Robert Munteanu
> Fix For: 1.8, 1.7.9
>
> Attachments: osgi-metadata-1.7.8.json, osgi-metadata-trunk.json
>
>
> We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi
> R6 annotations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6741) Switch to official OSGi component and metatype annotations
[ https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194264#comment-16194264 ] Julian Sedding commented on OAK-6741: - bq. Julian Sedding you might want to check your facts before pointing the finger. we already have a test for this [0]. why this test is not failing with the changed annotation, is a different question (it's probably the test framework that does not respect the wiring, but I'm no OSGi expert). With all due respect [~stillalex], I think your quoting me out of context. As for the facts, you say yourself that the test does not fail. So we don't have a test that covers the scenario imlemented in OAK-6753. This is evident by the fact that the test was not adjusted when OAK-6753 was implemented. Anyway, we're splitting hairs. If you would like me to revert the patch, please tell me so. > Switch to official OSGi component and metatype annotations > -- > > Key: OAK-6741 > URL: https://issues.apache.org/jira/browse/OAK-6741 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Robert Munteanu > Fix For: 1.8, 1.7.9 > > Attachments: osgi-metadata-1.7.8.json, osgi-metadata-trunk.json > > > We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi > R6 annotations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (OAK-6788) Build Jackrabbit Oak #842 failed
Hudson created OAK-6788: --- Summary: Build Jackrabbit Oak #842 failed Key: OAK-6788 URL: https://issues.apache.org/jira/browse/OAK-6788 Project: Jackrabbit Oak Issue Type: Bug Components: continuous integration Reporter: Hudson No description is provided The build Jackrabbit Oak #842 has failed. First failed run: [Jackrabbit Oak #842|https://builds.apache.org/job/Jackrabbit%20Oak/842/] [console log|https://builds.apache.org/job/Jackrabbit%20Oak/842/console] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6753) Wrong binding in TokenConfigurationImpl
[
https://issues.apache.org/jira/browse/OAK-6753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194245#comment-16194245
]
Alex Deparvu commented on OAK-6753:
---
with rev http://svn.apache.org/viewvc?rev=1811297=rev I optimized the case
when there's a single entry in the map (the default one). thanks [~asanso] for
the review.
> Wrong binding in TokenConfigurationImpl
>
>
> Key: OAK-6753
> URL: https://issues.apache.org/jira/browse/OAK-6753
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Reporter: Antonio Sanso
>Assignee: Alex Deparvu
> Fix For: 1.7.9
>
>
> OAK-6741 made a Switch to official OSGi component and metatype annotations.
> I spotted at least one lost in translation issue.
> {code}
> -@References({
> -@Reference(
> -name = "credentialsSupport",
> -referenceInterface = CredentialsSupport.class,
> -cardinality = ReferenceCardinality.OPTIONAL_MULTIPLE,
> -policy = ReferencePolicy.DYNAMIC)
> -})
> +@Reference(name = "credentialsSupport",
> +cardinality = ReferenceCardinality.OPTIONAL,
> +policy = ReferencePolicy.DYNAMIC)
> +@SuppressWarnings("UnusedDeclaration")
> {code}
> the {{CredentialsSupport}} reference used to be {{cardinality =
> ReferenceCardinality.OPTIONAL_MULTIPLE}} but is {{cardinality =
> ReferenceCardinality.OPTIONAL}} after the "migration"
> cc [~stillalex] [~jsedding] [~anchela]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6318) Refactor oak.spi.security into a separate module/bundle
[
https://issues.apache.org/jira/browse/OAK-6318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194229#comment-16194229
]
angela commented on OAK-6318:
-
[~rombert], thanks for your input. regarding:
{quote}
but maybe copy the affected classes to a new package and a new bundle would
solve the issue
{quote}
That's what we did with the various subtasks in OAK-6319. However, the main
issue doesn't get addressed by that. Which is: we cannot move the entire
_o.a.j.oak.plugins.tree_ package including the deprecated factories to
_oak-security-spi_ without introducing cyclic dependencies to _oak-core_... and
we tried more than once. We felt that we only had 2 options left:
- splitting the _o.a.j.oak.plugins.tree_ package across different bundles (i.e.
keeping the factories in oak-core while moving the rest to oak-security-spi as
[~stillalex] did... that worked for us locally but caused failures on some
platforms.
- major version bump by entirely dropping the factories from
_o.a.j.oak.plugins.tree_ (we already have the replacement and refactored all
usages in oak, see OAK-6319)... so basically that is what is left at this point.
If there is a option we missed to evaluate so far... even better.
> Refactor oak.spi.security into a separate module/bundle
> ---
>
> Key: OAK-6318
> URL: https://issues.apache.org/jira/browse/OAK-6318
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: core, security
>Reporter: angela
>Assignee: angela
> Attachments: PropInfo.patch
>
>
> In the light of the modularization effort it would IMHO make a lot of sense
> to refactor _oak.spi.security.*_ into a separate module/bundle that could be
> release independent of _oak-core_.
> As far as I could see so far this is currently not easily doable due to the
> dependency of {{oak.spi.security.*}} on {{oak.plugins.tree}}, which in turn
> relies on {{o.a.j.oak.query.QueryEngineSettings}} and other oak-core
> internals (see also OAK-6304). Most likely this issue would therefore require
> a complete review (and possibly a split) of the {{oak.plugins.tree}} package
> space which contains a mixture of utilities and 'API' (like e.g.
> {{TreeContext}} and {{TreeLocation}})... will open a separate issue for this
> linking to this one.
> cc: [~stillalex], [~rombert], [~mduerig], [~tmueller]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (OAK-6787) Delete property index entries recursively in batches to avoid large transaction
[ https://issues.apache.org/jira/browse/OAK-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-6787: - Attachment: OAK-6787-v1.patch [patch|^OAK-6787-v1.patch] for the same (wiring to cleaner missing) [~mreutegg] [~catholicon] Please review. I was thinking to use this when NodeStore is of type Clusterable. wdyt? > Delete property index entries recursively in batches to avoid large > transaction > --- > > Key: OAK-6787 > URL: https://issues.apache.org/jira/browse/OAK-6787 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.8 > > Attachments: OAK-6787-v1.patch > > > PropertyIndexCleaner currently remove the property index bucket in a single > remove. This would work fine with SegmentNodeStore but would cause issue with > DocumentNodeStore where it would result in a one large commit. > To avoid this scenario we should implement recursive delete. This approach > would only be used if NodeStore implements Clusterable -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (OAK-6218) Including id in DocumentStoreException which wrap MongoException
[
https://issues.apache.org/jira/browse/OAK-6218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194224#comment-16194224
]
Chetan Mehrotra commented on OAK-6218:
--
[~mreutegg] This issue is seen quite a few times now. Would be good if we can
have this in 1.8
> Including id in DocumentStoreException which wrap MongoException
>
>
> Key: OAK-6218
> URL: https://issues.apache.org/jira/browse/OAK-6218
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: mongomk
>Reporter: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.8
>
>
> Currently when any exception occurs at Mongo level it gets wrapped in
> DocumentStoreException. To help in debugging such issues it would be good to
> also include the documentId(s) which was being processed in that call as part
> of exception message
> {noformat}
> Caused by: org.apache.jackrabbit.oak.plugins.document.DocumentStoreException:
> { "serverUsed" : "mongoserver:20001" , "ok" : 1 , "n" : 0 , "updatedExisting"
> : false , "err" : "Resulting document after update is larger than 16777216" ,
> "code" : 17419}
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentStoreException.convert(DocumentStoreException.java:48)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentStoreException.convert(DocumentStoreException.java:41)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.findAndModify(MongoDocumentStore.java:789)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createOrUpdate(MongoDocumentStore.java:805)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.createOrUpdate(MongoDocumentStore.java:884)
> at
> org.apache.jackrabbit.oak.plugins.document.util.LeaseCheckDocumentStoreWrapper.createOrUpdate(LeaseCheckDocumentStoreWrapper.java:133)
> at
> org.apache.jackrabbit.oak.plugins.document.Commit.applyToDocumentStore(Commit.java:308)
> at
> org.apache.jackrabbit.oak.plugins.document.Commit.applyToDocumentStore(Commit.java:245)
> at
> org.apache.jackrabbit.oak.plugins.document.Commit.applyInternal(Commit.java:214)
> at
> org.apache.jackrabbit.oak.plugins.document.Commit.apply(Commit.java:203)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.persist(DocumentNodeStoreBranch.java:292)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.persist(DocumentNodeStoreBranch.java:262)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.access$300(DocumentNodeStoreBranch.java:57)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch$InMemory.merge(DocumentNodeStoreBranch.java:499)
> at
> org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreBranch.merge0(DocumentNodeStoreBranch.java:182)
> ... 36 common frames omitted
> Caused by: com.mongodb.WriteConcernException: { "serverUsed" :
> "mongoserver:20001" , "ok" : 1 , "n" : 0 , "updatedExisting" : false , "err"
> : "Resulting document after update is larger than 16777216" , "code" : 17419}
> at com.mongodb.CommandResult.getWriteException(CommandResult.java:90)
> at com.mongodb.CommandResult.getException(CommandResult.java:79)
> at
> com.mongodb.DBCollectionImpl.translateBulkWriteException(DBCollectionImpl.java:414)
> at com.mongodb.DBCollectionImpl.updateImpl(DBCollectionImpl.java:292)
> at com.mongodb.DBCollection.update(DBCollection.java:250)
> at com.mongodb.DBCollection.update(DBCollection.java:232)
> at com.mongodb.DBCollection.update(DBCollection.java:307)
> at com.mongodb.DBCollection.update(DBCollection.java:322)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.findAndModify(MongoDocumentStore.java:746)
> ... 48 common frames omitted
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OAK-6741) Switch to official OSGi component and metatype annotations
[ https://issues.apache.org/jira/browse/OAK-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194212#comment-16194212 ] Alex Deparvu commented on OAK-6741: --- bq. However, I have to wonder, why is there no test coverage for OAK-6753?! [~jsedding] you might want to check your facts before pointing the finger. we already have a test for this [0]. why this test is not failing with the changed annotation, is a different question (it's probably the test framework that does not respect the wiring, but I'm no OSGi expert). [0] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/test/java/org/apache/jackrabbit/oak/security/authentication/token/TokenConfigurationImplOSGiTest.java#L74 > Switch to official OSGi component and metatype annotations > -- > > Key: OAK-6741 > URL: https://issues.apache.org/jira/browse/OAK-6741 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Robert Munteanu > Fix For: 1.8, 1.7.9 > > Attachments: osgi-metadata-1.7.8.json, osgi-metadata-trunk.json > > > We should remove the 'old' Felix SCR annotations and move to the 'new' OSGi > R6 annotations. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
