[jira] [Comment Edited] (OAK-4397) DefaultSyncContext.syncMembership may sync group of a foreign IDP

2016-10-07 Thread Alexander Klimetschek (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556027#comment-15556027
 ] 

Alexander Klimetschek edited comment on OAK-4397 at 10/7/16 7:44 PM:
-

Turns out, OAK-4845 is not a regression, as boundaries need to be established 
between multiple IDPs as well as possibly locally set memberships.

*Possible workarounds*

Existing applications relying on the previous behavior that allowed adding 
memberships to local groups (and these were really only intended as 
"placeholder" for the external groups) could do this:
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again

Also, in the general principal and access control design, the external group 
might be used solely for the representation of the memberships. A separate 
local "role" group would be used for defining the access control, and this one 
would include the external group as member, so the permissions get inherited to 
the external users that are member of that group.

Please note that this only affects external groups, but not the 
{{user.autoMembership}} setting of the DefaultSyncHandler.


was (Author: alexander.klimetschek):
Turns out, OAK-4845 is not a regression, as boundaries need to be established 
between multiple IDPs as well as possibly locally set memberships.

Possible workarounds for existing applications relying on the previous behavior 
that allowed adding memberships to local groups (and these were really only 
intended as "placeholder" for the external groups):
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again

Also, in the general principal and access control design, the external group 
might be used solely for the representation of the memberships. A separate 
"role" group would be used for defining the access control, and this one would 
include the external group as member, so the permissions get inherited to the 
external users that are member of that group.

Please note that this only affects external groups, but not the 
{{user.autoMembership}} setting of the DefaultSyncHandler.

> DefaultSyncContext.syncMembership may sync group of a foreign IDP
> -
>
> Key: OAK-4397
> URL: https://issues.apache.org/jira/browse/OAK-4397
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: auth-external
>Reporter: angela
>Assignee: angela
>Priority: Critical
>  Labels: security
> Fix For: 1.5.3, 1.4.7
>
>
> With the following scenario the {{DefaultSyncContext.syncMembership}} may end 
> up synchronizing (i.e. updating) a group defined by an foreign IDP and even 
> add the user to be synchronized as a new member:
> - configuration with different IDPs
> - foreign IDP synchronizes a given external group 'groupA' => rep:externalID 
> points to foreign-IDP (e.g. rep:externalId = 'groupA;foreignIDP')
> - my-IDP contains a group with the same ID (but obviously with a different 
> rep:externalID) and user that has declared group membership pointing to 
> 'groupA' from my IDP
> if synchronizing my user first the groupA will be created with a 
> rep:externalId = 'groupA;myIDP'.
> however, if the group has been synced before by the foreignIDP the code fails 
> to verify that an existing group 'groupA' really belongs to the same IDP and 
> thus may end up synchronizing the group and updating it's members.
> IMHO that's a critical issue as it violates the IDP boundaries.
> the fix is pretty trivial as it only requires testing for the IDP of the 
> existing group as we do it in other places (even in the same method).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4397) DefaultSyncContext.syncMembership may sync group of a foreign IDP

2016-10-07 Thread Alexander Klimetschek (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556027#comment-15556027
 ] 

Alexander Klimetschek edited comment on OAK-4397 at 10/7/16 7:42 PM:
-

Turns out, OAK-4845 is not a regression, as boundaries need to be established 
between multiple IDPs as well as possibly locally set memberships.

Possible workarounds for existing applications relying on the previous behavior 
that allowed adding memberships to local groups (and these were really only 
intended as "placeholder" for the external groups):
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again

Also, in the general principal and access control design, the external group 
might be used solely for the representation of the memberships. A separate 
"role" group would be used for defining the access control, and this one would 
include the external group as member, so the permissions get inherited to the 
external users that are member of that group.

Please note that this only affects external groups, but not the 
{{user.autoMembership}} setting of the DefaultSyncHandler.


was (Author: alexander.klimetschek):
Turns out, OAK-4845 is not a regression, as boundaries need to be established 
between multiple IDPs as well as possibly locally set memberships.

Possible workarounds for existing applications relying on the previous behavior 
that allowed adding memberships to local groups (and these were really only 
intended as "placeholder" for the external groups):
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again

> DefaultSyncContext.syncMembership may sync group of a foreign IDP
> -
>
> Key: OAK-4397
> URL: https://issues.apache.org/jira/browse/OAK-4397
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: auth-external
>Reporter: angela
>Assignee: angela
>Priority: Critical
>  Labels: security
> Fix For: 1.5.3, 1.4.7
>
>
> With the following scenario the {{DefaultSyncContext.syncMembership}} may end 
> up synchronizing (i.e. updating) a group defined by an foreign IDP and even 
> add the user to be synchronized as a new member:
> - configuration with different IDPs
> - foreign IDP synchronizes a given external group 'groupA' => rep:externalID 
> points to foreign-IDP (e.g. rep:externalId = 'groupA;foreignIDP')
> - my-IDP contains a group with the same ID (but obviously with a different 
> rep:externalID) and user that has declared group membership pointing to 
> 'groupA' from my IDP
> if synchronizing my user first the groupA will be created with a 
> rep:externalId = 'groupA;myIDP'.
> however, if the group has been synced before by the foreignIDP the code fails 
> to verify that an existing group 'groupA' really belongs to the same IDP and 
> thus may end up synchronizing the group and updating it's members.
> IMHO that's a critical issue as it violates the IDP boundaries.
> the fix is pretty trivial as it only requires testing for the IDP of the 
> existing group as we do it in other places (even in the same method).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4397) DefaultSyncContext.syncMembership may sync group of a foreign IDP

2016-10-07 Thread Alexander Klimetschek (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556027#comment-15556027
 ] 

Alexander Klimetschek edited comment on OAK-4397 at 10/7/16 7:39 PM:
-

Turns out, OAK-4845 is not a regression, as boundaries need to be established 
between multiple IDPs as well as possibly locally set memberships.

Possible workarounds for existing applications relying on the previous behavior 
that allowed adding memberships to local groups (and these were really only 
intended as "placeholder" for the external groups):
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again


was (Author: alexander.klimetschek):
Turns out, OAK-4845 is not a regression, as it would only get more complex down 
the line.

Possible workarounds for existing applications relying on the previous behavior 
that allowed adding memberships to local groups (and these were really only 
intended as "placeholder" for the external groups):
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again

> DefaultSyncContext.syncMembership may sync group of a foreign IDP
> -
>
> Key: OAK-4397
> URL: https://issues.apache.org/jira/browse/OAK-4397
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: auth-external
>Reporter: angela
>Assignee: angela
>Priority: Critical
>  Labels: security
> Fix For: 1.5.3, 1.4.7
>
>
> With the following scenario the {{DefaultSyncContext.syncMembership}} may end 
> up synchronizing (i.e. updating) a group defined by an foreign IDP and even 
> add the user to be synchronized as a new member:
> - configuration with different IDPs
> - foreign IDP synchronizes a given external group 'groupA' => rep:externalID 
> points to foreign-IDP (e.g. rep:externalId = 'groupA;foreignIDP')
> - my-IDP contains a group with the same ID (but obviously with a different 
> rep:externalID) and user that has declared group membership pointing to 
> 'groupA' from my IDP
> if synchronizing my user first the groupA will be created with a 
> rep:externalId = 'groupA;myIDP'.
> however, if the group has been synced before by the foreignIDP the code fails 
> to verify that an existing group 'groupA' really belongs to the same IDP and 
> thus may end up synchronizing the group and updating it's members.
> IMHO that's a critical issue as it violates the IDP boundaries.
> the fix is pretty trivial as it only requires testing for the IDP of the 
> existing group as we do it in other places (even in the same method).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4397) DefaultSyncContext.syncMembership may sync group of a foreign IDP

2016-10-07 Thread Alexander Klimetschek (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556027#comment-15556027
 ] 

Alexander Klimetschek edited comment on OAK-4397 at 10/7/16 7:25 PM:
-

Turns out, OAK-4845 is not a regression, as it would only get more complex down 
the line.

Possible workarounds for existing applications relying on the previous behavior 
that allowed adding memberships to local groups (and these were really only 
intended as "placeholder" for the external groups):
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again


was (Author: alexander.klimetschek):
Turns out, OAK-4845 is not a regression, as it would only get more complex down 
the line.

Removing membership from local groups could create problems, if both a local 
and external group accidentally have the same name. A completely separate, 
locally established group membership should not be inadvertently be removed by 
a sync. Furthermore, supporting adding memberships to local groups on one hand, 
but not removing, creates an inconsistency and would not provide the guarantee 
expected from the sync, that the group memberships in the JCR represent the 
ones from the external IDP.

Possible workarounds for existing applications relying on the previous behavior 
that allowed adding memberships, and had local groups that really are only used 
as placeholder for the external groups:
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again

> DefaultSyncContext.syncMembership may sync group of a foreign IDP
> -
>
> Key: OAK-4397
> URL: https://issues.apache.org/jira/browse/OAK-4397
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: auth-external
>Reporter: angela
>Assignee: angela
>Priority: Critical
>  Labels: security
> Fix For: 1.5.3, 1.4.7
>
>
> With the following scenario the {{DefaultSyncContext.syncMembership}} may end 
> up synchronizing (i.e. updating) a group defined by an foreign IDP and even 
> add the user to be synchronized as a new member:
> - configuration with different IDPs
> - foreign IDP synchronizes a given external group 'groupA' => rep:externalID 
> points to foreign-IDP (e.g. rep:externalId = 'groupA;foreignIDP')
> - my-IDP contains a group with the same ID (but obviously with a different 
> rep:externalID) and user that has declared group membership pointing to 
> 'groupA' from my IDP
> if synchronizing my user first the groupA will be created with a 
> rep:externalId = 'groupA;myIDP'.
> however, if the group has been synced before by the foreignIDP the code fails 
> to verify that an existing group 'groupA' really belongs to the same IDP and 
> thus may end up synchronizing the group and updating it's members.
> IMHO that's a critical issue as it violates the IDP boundaries.
> the fix is pretty trivial as it only requires testing for the IDP of the 
> existing group as we do it in other places (even in the same method).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4397) DefaultSyncContext.syncMembership may sync group of a foreign IDP

2016-10-07 Thread Alexander Klimetschek (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556027#comment-15556027
 ] 

Alexander Klimetschek commented on OAK-4397:


Turns out, OAK-4845 is not a regression, as it would only get more complex down 
the line.

Removing membership from local groups could create problems, if both a local 
and external group accidentally have the same name. A completely separate, 
locally established group membership should not be inadvertently be removed by 
a sync. Furthermore, supporting adding memberships to local groups on one hand, 
but not removing, creates an inconsistency and would not provide the guarantee 
expected from the sync, that the group memberships in the JCR represent the 
ones from the external IDP.

Possible workarounds for existing applications relying on the previous behavior 
that allowed adding memberships, and had local groups that really are only used 
as placeholder for the external groups:
* right before a batch sync is run, remove the local group and get it 
automatically recreated using the right {{rep:externalId}} (if temporarily 
loosing the group has no bad consequences; note that ACs would not be affected)
* temporarily disable the {{rep:externalId}} protection 
({{protectExternalId=false}}, see 
[docs|http://jackrabbit.apache.org/oak/docs/security/authentication/external/defaultusersync.html#Configuration_of_the_ExternalPrincipalConfiguration]),
 set the correct {{rep:externalId}} values on existing groups, enable 
protection again

> DefaultSyncContext.syncMembership may sync group of a foreign IDP
> -
>
> Key: OAK-4397
> URL: https://issues.apache.org/jira/browse/OAK-4397
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: auth-external
>Reporter: angela
>Assignee: angela
>Priority: Critical
>  Labels: security
> Fix For: 1.5.3, 1.4.7
>
>
> With the following scenario the {{DefaultSyncContext.syncMembership}} may end 
> up synchronizing (i.e. updating) a group defined by an foreign IDP and even 
> add the user to be synchronized as a new member:
> - configuration with different IDPs
> - foreign IDP synchronizes a given external group 'groupA' => rep:externalID 
> points to foreign-IDP (e.g. rep:externalId = 'groupA;foreignIDP')
> - my-IDP contains a group with the same ID (but obviously with a different 
> rep:externalID) and user that has declared group membership pointing to 
> 'groupA' from my IDP
> if synchronizing my user first the groupA will be created with a 
> rep:externalId = 'groupA;myIDP'.
> however, if the group has been synced before by the foreignIDP the code fails 
> to verify that an existing group 'groupA' really belongs to the same IDP and 
> thus may end up synchronizing the group and updating it's members.
> IMHO that's a critical issue as it violates the IDP boundaries.
> the fix is pretty trivial as it only requires testing for the IDP of the 
> existing group as we do it in other places (even in the same method).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (OAK-4265) XPath: support limited form of "union"

2016-10-07 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller resolved OAK-4265.
-
Resolution: Fixed

http://svn.apache.org/r1763760 (documentation)

> XPath: support limited form of "union"
> --
>
> Key: OAK-4265
> URL: https://issues.apache.org/jira/browse/OAK-4265
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
> Fix For: 1.5.12
>
>
> Currently, SQL-2 supports multiple paths restrictions combined with "or", and 
> it supports "union". XPath doesn't support either. It would be good if this 
> is supported, specially the "multiple paths" part.
> I think it makes sense to convert this to a SQL-2 "union" in all cases, even 
> if it's just multiple paths.
> It doesn't need to be a "generic" solution, so that "|" doesn't need to be 
> supported at every place (for example in the condition, or in the "order 
> by"). But it would be nice if multiple independent queries, with one order 
> by, could be supported.
> Implementation note: what makes this a bit complicated is support for "order 
> by".
> For example:
> {noformat}
> /jcr:root/(content|lib)/element(*, nt:base) order by @title
> select [jcr:path], [jcr:score], * 
> from [nt:base] as a 
> where ischildnode(a, '/content') 
> /* xpath: /jcr:root/content/element(*, nt:base) order by @title */ 
> union select [jcr:path], [jcr:score], * 
> from [nt:base] as a 
> where ischildnode(a, '/lib') 
> /* xpath: /jcr:root/lib/element(*, nt:base) order by @title */ 
> order by [title]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4265) XPath: support limited form of "union"

2016-10-07 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller updated OAK-4265:

Fix Version/s: (was: 1.6)
   1.5.12

> XPath: support limited form of "union"
> --
>
> Key: OAK-4265
> URL: https://issues.apache.org/jira/browse/OAK-4265
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
> Fix For: 1.5.12
>
>
> Currently, SQL-2 supports multiple paths restrictions combined with "or", and 
> it supports "union". XPath doesn't support either. It would be good if this 
> is supported, specially the "multiple paths" part.
> I think it makes sense to convert this to a SQL-2 "union" in all cases, even 
> if it's just multiple paths.
> It doesn't need to be a "generic" solution, so that "|" doesn't need to be 
> supported at every place (for example in the condition, or in the "order 
> by"). But it would be nice if multiple independent queries, with one order 
> by, could be supported.
> Implementation note: what makes this a bit complicated is support for "order 
> by".
> For example:
> {noformat}
> /jcr:root/(content|lib)/element(*, nt:base) order by @title
> select [jcr:path], [jcr:score], * 
> from [nt:base] as a 
> where ischildnode(a, '/content') 
> /* xpath: /jcr:root/content/element(*, nt:base) order by @title */ 
> union select [jcr:path], [jcr:score], * 
> from [nt:base] as a 
> where ischildnode(a, '/lib') 
> /* xpath: /jcr:root/lib/element(*, nt:base) order by @title */ 
> order by [title]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554801#comment-15554801
 ] 

Julian Reschke edited comment on OAK-4905 at 10/7/16 1:03 PM:
--

trunk: [r1763735|http://svn.apache.org/r1763735]
1.4: [r1763744|http://svn.apache.org/r1763744]
1.2: [r1763753|http://svn.apache.org/r1763753]
1.0: [r1763758|http://svn.apache.org/r1763758]



was (Author: reschke):
trunk: [r1763735|http://svn.apache.org/r1763735]
1.4: [r1763744|http://svn.apache.org/r1763744]
1.2: [r1763753|http://svn.apache.org/r1763753]

> RDB*Store: update postgresql JDBC driver reference to 9.4.1211
> --
>
> Key: OAK-4905
> URL: https://issues.apache.org/jira/browse/OAK-4905
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: documentmk, rdbmk
>Affects Versions: 1.2.19, 1.0.34, 1.4.8, 1.5.11
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.2.20, 1.5.12, 1.0.35, 1.4.9
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4905:

Fix Version/s: 1.0.35

> RDB*Store: update postgresql JDBC driver reference to 9.4.1211
> --
>
> Key: OAK-4905
> URL: https://issues.apache.org/jira/browse/OAK-4905
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: documentmk, rdbmk
>Affects Versions: 1.2.19, 1.0.34, 1.4.8, 1.5.11
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.2.20, 1.5.12, 1.0.35, 1.4.9
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Thomas Mueller (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554980#comment-15554980
 ] 

Thomas Mueller commented on OAK-4904:
-

http://svn.apache.org/r1763756 (trunk)

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Thomas Mueller
>  Labels: performance
> Fix For: 1.5.12
>
> Attachments: OAK-4904-v1.patch, OAK-4904-v2.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller resolved OAK-4904.
-
Resolution: Fixed

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Thomas Mueller
>  Labels: performance
> Fix For: 1.5.12
>
> Attachments: OAK-4904-v1.patch, OAK-4904-v2.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller reassigned OAK-4904:
---

Assignee: Thomas Mueller  (was: Chetan Mehrotra)

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Thomas Mueller
>  Labels: performance
> Fix For: 1.5.12
>
> Attachments: OAK-4904-v1.patch, OAK-4904-v2.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller updated OAK-4904:

Fix Version/s: (was: 1.6)
   1.5.12

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Thomas Mueller
>  Labels: performance
> Fix For: 1.5.12
>
> Attachments: OAK-4904-v1.patch, OAK-4904-v2.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554801#comment-15554801
 ] 

Julian Reschke edited comment on OAK-4905 at 10/7/16 12:18 PM:
---

trunk: [r1763735|http://svn.apache.org/r1763735]
1.4: [r1763744|http://svn.apache.org/r1763744]
1.2: [r1763753|http://svn.apache.org/r1763753]


was (Author: reschke):
trunk: [r1763735|http://svn.apache.org/r1763735]
1.4: [r1763744|http://svn.apache.org/r1763744]

> RDB*Store: update postgresql JDBC driver reference to 9.4.1211
> --
>
> Key: OAK-4905
> URL: https://issues.apache.org/jira/browse/OAK-4905
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: documentmk, rdbmk
>Affects Versions: 1.2.19, 1.0.34, 1.4.8, 1.5.11
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.2.20, 1.5.12, 1.4.9
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4905:

Fix Version/s: 1.2.20

> RDB*Store: update postgresql JDBC driver reference to 9.4.1211
> --
>
> Key: OAK-4905
> URL: https://issues.apache.org/jira/browse/OAK-4905
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: documentmk, rdbmk
>Affects Versions: 1.2.19, 1.0.34, 1.4.8, 1.5.11
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.2.20, 1.5.12, 1.4.9
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Thomas Mueller (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller updated OAK-4904:

Attachment: OAK-4904-v2.patch

OAK-4904-v2.patch
* the cost for "is not null" and "in(a, b, c)" conditions is unchanged; just "x 
= y" is affected
* added tests

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>  Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4904-v1.patch, OAK-4904-v2.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554801#comment-15554801
 ] 

Julian Reschke edited comment on OAK-4905 at 10/7/16 11:43 AM:
---

trunk: [r1763735|http://svn.apache.org/r1763735]
1.4: [r1763744|http://svn.apache.org/r1763744]


was (Author: reschke):
trunk: [r1763735|http://svn.apache.org/r1763735]


> RDB*Store: update postgresql JDBC driver reference to 9.4.1211
> --
>
> Key: OAK-4905
> URL: https://issues.apache.org/jira/browse/OAK-4905
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: documentmk, rdbmk
>Affects Versions: 1.2.19, 1.0.34, 1.4.8, 1.5.11
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.5.12, 1.4.9
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4905:

Fix Version/s: 1.4.9

> RDB*Store: update postgresql JDBC driver reference to 9.4.1211
> --
>
> Key: OAK-4905
> URL: https://issues.apache.org/jira/browse/OAK-4905
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: documentmk, rdbmk
>Affects Versions: 1.2.19, 1.0.34, 1.4.8, 1.5.11
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.5.12, 1.4.9
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Thomas Mueller (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554842#comment-15554842
 ] 

Thomas Mueller commented on OAK-4904:
-

There are some smaller changes I would like to add: for "is not null" and "x 
in(a, b, c)" another index might be faster. I will attach a new patch as soon 
as I'm done.

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>  Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4904-v1.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554801#comment-15554801
 ] 

Julian Reschke commented on OAK-4905:
-

trunk: [r1763735|http://svn.apache.org/r1763735]


> RDB*Store: update postgresql JDBC driver reference to 9.4.1211
> --
>
> Key: OAK-4905
> URL: https://issues.apache.org/jira/browse/OAK-4905
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: documentmk, rdbmk
>Affects Versions: 1.2.19, 1.0.34, 1.4.8, 1.5.11
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.5.12
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke resolved OAK-4905.
-
   Resolution: Fixed
Fix Version/s: 1.5.12

> RDB*Store: update postgresql JDBC driver reference to 9.4.1211
> --
>
> Key: OAK-4905
> URL: https://issues.apache.org/jira/browse/OAK-4905
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: documentmk, rdbmk
>Affects Versions: 1.2.19, 1.0.34, 1.4.8, 1.5.11
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.5.12
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4870) Implement caching for S3DataStore

2016-10-07 Thread Amit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554786#comment-15554786
 ] 

Amit Jain commented on OAK-4870:


The patch available at 
https://github.com/amit-jain/jackrabbit-oak/compare/trunk...amit-jain:OAK-4837.
Patch creates a new implementation and has a flag (enabled by default) to 
switch to older implementation if needed for now.

> Implement caching for S3DataStore
> -
>
> Key: OAK-4870
> URL: https://issues.apache.org/jira/browse/OAK-4870
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
>
> S3DataStore would be required to be updated to make use of the improved 
> caching.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4837) Improved caching for DataStore

2016-10-07 Thread Amit Jain (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554782#comment-15554782
 ] 

Amit Jain commented on OAK-4837:


Reworked caching work done at 
https://github.com/amit-jain/jackrabbit-oak/compare/trunk...amit-jain:OAK-4837
The patch is fully tested including running on Adobe AEM.

> Improved caching for DataStore
> --
>
> Key: OAK-4837
> URL: https://issues.apache.org/jira/browse/OAK-4837
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
>  Labels: performance
>
> The current CachingDataStore implementation used with S3DataStore has certain 
> problems:
> * Lack of stats to show hit rate/miss rates for files being requested for 
> downloads
> * Lack of stats for async uploads
> * CachingDataStore starts proactively downloading files in the background 
> when a call to {{getRecord}} is made.
> * Async upload functionality leaks into Backend implementations, LocalCache 
> classes.
> * The call to {{DataStore#getRecord()}} which makes multiple calls to 
> backends which is problematic for S3 (i.e. when not being served bu cache)
> * There is some functionality which is not used with Oak like length cache, 
> sync/async touch etc. which can be removed and code simplified.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4837) Improved caching for DataStore

2016-10-07 Thread Amit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amit Jain updated OAK-4837:
---
Description: 
The current CachingDataStore implementation used with S3DataStore has certain 
problems:
* Lack of stats to show hit rate/miss rates for files being requested for 
downloads
* Lack of stats for async uploads
* CachingDataStore starts proactively downloading files in the background when 
a call to {{getRecord}} is made.
* Async upload functionality leaks into Backend implementations, LocalCache 
classes.
* The call to {{DataStore#getRecord()}} which makes multiple calls to backends 
which is problematic for S3 (i.e. when not being served bu cache)
* There is some functionality which is not used with Oak like length cache, 
sync/async touch etc. which can be removed and code simplified.

  was:
The current CachingDataStore implementation used with S3DataStore has certain 
problems:
* Lack of stats to show hit rate/miss rates for files being requested for 
downloads
* Lack of stats for async uploads
* Async upload functionality leaks into Backend implementations, LocalCache 
classes.
* The call to {{DataStore#getRecord()}} which makes multiple calls to backends 
which is problematic for S3 (i.e. when not being served bu cache)
* There is some functionality which is not used with Oak like length cache, 
sync/async touch etc. which can be removed and code simplified.


> Improved caching for DataStore
> --
>
> Key: OAK-4837
> URL: https://issues.apache.org/jira/browse/OAK-4837
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
>  Labels: performance
>
> The current CachingDataStore implementation used with S3DataStore has certain 
> problems:
> * Lack of stats to show hit rate/miss rates for files being requested for 
> downloads
> * Lack of stats for async uploads
> * CachingDataStore starts proactively downloading files in the background 
> when a call to {{getRecord}} is made.
> * Async upload functionality leaks into Backend implementations, LocalCache 
> classes.
> * The call to {{DataStore#getRecord()}} which makes multiple calls to 
> backends which is problematic for S3 (i.e. when not being served bu cache)
> * There is some functionality which is not used with Oak like length cache, 
> sync/async touch etc. which can be removed and code simplified.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (OAK-4905) RDB*Store: update postgresql JDBC driver reference to 9.4.1211

2016-10-07 Thread Julian Reschke (JIRA)
Julian Reschke created OAK-4905:
---

 Summary: RDB*Store: update postgresql JDBC driver reference to 
9.4.1211
 Key: OAK-4905
 URL: https://issues.apache.org/jira/browse/OAK-4905
 Project: Jackrabbit Oak
  Issue Type: Technical task
Affects Versions: 1.5.11, 1.4.8, 1.0.34, 1.2.19
Reporter: Julian Reschke
Assignee: Julian Reschke
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Chetan Mehrotra (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15554678#comment-15554678
 ] 

Chetan Mehrotra edited comment on OAK-4904 at 10/7/16 9:52 AM:
---

[patch|^OAK-4904-v1.patch] for the same. The test fails on 1.4 also 

[~tmueller] - Please review the patch. There is TODO left around should be 
remove the count calculation altogether for unique indexes. Fix used here is 
return the minimum cost in case query can be handled by a unique index


was (Author: chetanm):
[patch|^OAK-4904-v1.patch] for the same. The test fails on 1.4 also 

[~tmueller] - Please review the patch. There is TODO left around should be 
remove the count calculation altogether for unique indexes 

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>  Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4904-v1.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra updated OAK-4904:
-
Labels: performance  (was: )

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>  Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4904-v1.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Chetan Mehrotra (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chetan Mehrotra updated OAK-4904:
-
Attachment: OAK-4904-v1.patch

[patch|^OAK-4904-v1.patch] for the same. The test fails on 1.4 also 

[~tmueller] - Please review the patch. There is TODO left around should be 
remove the count calculation altogether for unique indexes 

> For unique indexes avoid consulting indexes other than property index
> -
>
> Key: OAK-4904
> URL: https://issues.apache.org/jira/browse/OAK-4904
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>  Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4904-v1.patch
>
>
> Currently for queries involving unique index like 
> {noformat}
> SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
> {noformat}
> QueryEngine would start consulting all types of index in order of minimum 
> cost. Where minimum cost are
> * PropertyIndex - 2.0
> * LucenePropertyIndex - 2.1
> For such queries in case of match property index returns a cost of (base) 2 + 
> (count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
> queries a very frequent it would be better to avoid consulting Lucene index 
> as that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (OAK-4904) For unique indexes avoid consulting indexes other than property index

2016-10-07 Thread Chetan Mehrotra (JIRA)
Chetan Mehrotra created OAK-4904:


 Summary: For unique indexes avoid consulting indexes other than 
property index
 Key: OAK-4904
 URL: https://issues.apache.org/jira/browse/OAK-4904
 Project: Jackrabbit Oak
  Issue Type: Improvement
  Components: query
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
 Fix For: 1.6


Currently for queries involving unique index like 

{noformat}
SELECT * FROM [nt:base] WHERE [jcr:uuid] = $id
{noformat}

QueryEngine would start consulting all types of index in order of minimum cost. 
Where minimum cost are
* PropertyIndex - 2.0
* LucenePropertyIndex - 2.1

For such queries in case of match property index returns a cost of (base) 2 + 
(count) 1 = 3 due to which QE has to consult Lucene index also. Given such 
queries a very frequent it would be better to avoid consulting Lucene index as 
that adds unnecessary overhead



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (OAK-4048) [regression] SyncHandler.listIdentities() returns all users, not only external ones

2016-10-07 Thread Amit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amit Jain closed OAK-4048.
--

> [regression] SyncHandler.listIdentities() returns all users, not only 
> external ones
> ---
>
> Key: OAK-4048
> URL: https://issues.apache.org/jira/browse/OAK-4048
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: auth-external
>Reporter: Alexander Klimetschek
>Assignee: Tobias Bocanegra
> Fix For: 1.6, 1.4.8
>
> Attachments: OAK-4048.patch
>
>
> OAK-3508 accidentally introduced a regression in 
> https://github.com/apache/jackrabbit-oak/commit/cc78f6fdd122d1c9f200b43fc2b9536518ea996b#diff-490ff25c104d019ee25f92b2b8bdbabd
> If {{getIdentityRef(auth)}} returns null, {{createSyncedIdentity}} needs to 
> return null.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (OAK-4811) MongoToMongoFbsTest fails

2016-10-07 Thread Amit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amit Jain closed OAK-4811.
--

> MongoToMongoFbsTest fails
> -
>
> Key: OAK-4811
> URL: https://issues.apache.org/jira/browse/OAK-4811
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: upgrade
>Affects Versions: 1.4.7
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
> Fix For: 1.4.8
>
>
> The test fails in the current 1.4 branch and also with the 1.4.7 release when 
> a local MongoDB is running.
> {noformat}
> validateMigration(org.apache.jackrabbit.oak.upgrade.cli.MongoToMongoFbsTest)  
> Time elapsed: 5.628 sec  <<< ERROR!
> java.lang.IllegalStateException: This builder does not exist: default
> {noformat}
> The test runs successfully with 1.4.6.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (OAK-4821) Allow use of Java 7 in Oak 1.4

2016-10-07 Thread Amit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amit Jain closed OAK-4821.
--

> Allow use of Java 7 in Oak 1.4
> --
>
> Key: OAK-4821
> URL: https://issues.apache.org/jira/browse/OAK-4821
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: parent
>Affects Versions: 1.4.7
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Fix For: 1.4.8
>
>
> (mailing list discussion in progress)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (OAK-4174) SegmentToJdbcTest failing with improvements of OAK-4119

2016-10-07 Thread Amit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amit Jain closed OAK-4174.
--

> SegmentToJdbcTest failing with improvements of  OAK-4119
> 
>
> Key: OAK-4174
> URL: https://issues.apache.org/jira/browse/OAK-4174
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: upgrade
>Reporter: angela
>Assignee: Tomek Rękawek
>Priority: Critical
> Fix For: 1.6, 1.4.8
>
>
> Despite the fact that  OAK-4128 has been fixed I get a test failure for 
> SegmentToJdbcTest.validateMigration with my pending improvements from 
> OAK-4119.
> In order not to break the build I will tmp mark the test with @Ignore as 
> discussed on the mailing list.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4860) Backport OAK-4301 and OAK-4825

2016-10-07 Thread Amit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amit Jain updated OAK-4860:
---
Fix Version/s: (was: 1.4.8)

> Backport OAK-4301 and OAK-4825
> --
>
> Key: OAK-4860
> URL: https://issues.apache.org/jira/browse/OAK-4860
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: auth-external, core
>Reporter: Dominique Jäggi
>Assignee: Dominique Jäggi
> Fix For: 1.4.9
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4901) Improve SizeDeltaGcEstimation logging

2016-10-07 Thread Valentin Olteanu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552536#comment-15552536
 ] 

Valentin Olteanu edited comment on OAK-4901 at 10/7/16 7:52 AM:


[~alex.parvulescu] could you please review the attached patch? Thanks! 

PS: I've just realized the patch has an unwanted change in {{pom.xml}} which 
should be ignored.


was (Author: volteanu):
[~alex.parvulescu] could you please review the attached patch? Thanks! 

> Improve SizeDeltaGcEstimation logging
> -
>
> Key: OAK-4901
> URL: https://issues.apache.org/jira/browse/OAK-4901
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segment-tar
>Affects Versions: Segment Tar 0.0.14
>Reporter: Valentin Olteanu
>Priority: Minor
> Attachments: OAK-4901.patch
>
>
> The current message logged after estimation is not very clear for someone 
> that does not know how it works:
> {code}
> 06.10.2016 18:53:07.819 *INFO* [TarMK revision gc 
> [/Users/volteanu/workspace/test/author/crx-quickstart/repository/segmentstore]]
>  org.apache.jackrabbit.oak.segment.file.FileStore TarMK GC #1: estimation 
> completed in 7.362 ms (7 ms). Size delta is 0% or 880.3 MB/880.9 MB 
> (880258560/880892416 bytes), so skipping compaction for now
> {code}
> I think it should also list the delta in absolute value and the threshold it 
> is compared against.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (OAK-4903) Async uploads in S3 causes issues in a cluster

2016-10-07 Thread Amit Jain (JIRA)
Amit Jain created OAK-4903:
--

 Summary: Async uploads in S3 causes issues in a cluster
 Key: OAK-4903
 URL: https://issues.apache.org/jira/browse/OAK-4903
 Project: Jackrabbit Oak
  Issue Type: Improvement
Reporter: Amit Jain
Assignee: Amit Jain


S3DataStore and CachingFDS through the CachingDataStore enable async uploads. 
This causes problems in clustered setups where uploads can sometimes be visible 
after a delay. During this time any request for the corresponding asset/file 
would return errors.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)