[jira] [Commented] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
[ https://issues.apache.org/jira/browse/OAK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505661#comment-15505661 ] Alexander Klimetschek commented on OAK-4825: The linger time is nice, but wouldn't necessarily be required in our use case, also considering it is some extra work (e.g. some new background job). What we currently do is a daily sync run that basically does syncAllExternalUsers() + purgeOrphanedUses() in SyncMbean terms. With the exception that we optimize it by calling list users and fetching of each external user only once. So it might be useful to have this as e.g. syncAndPurgeExternalUsers() on the Mbean or on the synchandler. > Support disabling of users instead of removal in DefaultSyncHandler > --- > > Key: OAK-4825 > URL: https://issues.apache.org/jira/browse/OAK-4825 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: auth-external >Reporter: Alexander Klimetschek > > The DefaultSyncHandler by default will remove (local) users when they are no > longer active in the external system aka no longer provided by the > ExternalIdentityProvider. It would be useful to have an option to _disable_ > users instead of removing them completely. > This is good for use cases that need to keep the user's data around in the > JCR and can't just delete it. Also, we have seen cases where the user is only > temporarily removed from the external identity system (e.g. accidentally > removed from group that maps them to the JCR system and quickly added back), > where a full removal can do harm. > (Note: There is an [option in the SyncContext > interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38] > to suppress purging, and the JMX sync commands such as > [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256] > "use" it. However, the JCR users look like "valid" users then locally. Even > if the authentication is done completely through the IDP and will fail > properly for these missing users, it can be difficult for other uses like > administration, monitoring, other application code to detect that such a user > is not active anymore.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4826) Auto removal of orphaned checkpoints
[
https://issues.apache.org/jira/browse/OAK-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505577#comment-15505577
]
Chetan Mehrotra edited comment on OAK-4826 at 9/20/16 5:16 AM:
---
[~alex.parvulescu] [~mreutegg] thoughts? I think this would be useful both
stores and reduce the dependency on manual steps
was (Author: chetanm):
[~alex.parvulescu] [~mreutegg] thoughts?
> Auto removal of orphaned checkpoints
>
>
> Key: OAK-4826
> URL: https://issues.apache.org/jira/browse/OAK-4826
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Chetan Mehrotra
> Fix For: 1.6
>
>
> Currently if in a running system there are some orphaned checkpoint present
> then they prevent the revision gc (compaction for segment) from being
> effective.
> So far the practice has been to use {{oak-run checkpoints rm-unreferenced}}
> command to clean them up manually. This was set to manual as it was not
> possible to determine whether current checkpoint is in use or not.
> rm-unreferenced works with the basis that checkpoints are only made from
> AsyncIndexUpdate and hence can check if the checkpoint is in use by cross
> checking with {{:async}} state. Doing it in auto mode is risky as
> {{checkpoint}} api can be used by any module.
> With OAK-2314 we also record some metadata like {{creator}} and {{name}}.
> This can be used for auto cleanup. For example in some running system
> following checkpoints are listed
> {noformat}
> Mon Sep 19 18:02:09 EDT 2016 Sun Jun 16 18:02:09 EDT 2019
> r15744787d0a-1-1
>
> creator=AsyncIndexUpdate
> name=fulltext-async
> thread=sling-default-4070-Registered Service.653
>
> Mon Sep 19 18:02:09 EDT 2016 Sun Jun 16 18:02:09 EDT 2019
> r15744787d0a-0-1
>
> creator=AsyncIndexUpdate
> name=async
> thread=sling-default-4072-Registered Service.656
>
> Fri Aug 19 18:57:33 EDT 2016 Thu May 16 18:57:33 EDT 2019
> r156a50612e1-1-1
>
> creator=AsyncIndexUpdate
> name=async
> thread=sling-default-10-Registered Service.654
>
> Wed Aug 10 12:13:20 EDT 2016 Tue May 07 12:25:52 EDT 2019
> r156753ac38d-0-1
>
> creator=AsyncIndexUpdate
> name=async
> thread=sling-default-6041-Registered Service.1966
> {noformat}
> As can be seen that last 2 checkpoints are orphan and they would prevent
> revision gc. For auto mode we can use following heuristic
> # List all current checkpoints
> # Only keep the latest checkpoint for given {{creator}} and {{name}} combo.
> Other entries from same pair which are older i.e. creation time can be
> consider orphan and deleted
> This logic can be implemented
> {{org.apache.jackrabbit.oak.checkpoint.Checkpoints}} and can be invoked by
> Revision GC logic (both in DocumentNodeStore and SegmentNodeStore) to
> determine the base revision to keep
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4552) Checkpoints.getOldestRevisionToKeep() returns newest valid revision
[ https://issues.apache.org/jira/browse/OAK-4552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505582#comment-15505582 ] Chetan Mehrotra commented on OAK-4552: -- While backporting this also consider OAK-4826 > Checkpoints.getOldestRevisionToKeep() returns newest valid revision > --- > > Key: OAK-4552 > URL: https://issues.apache.org/jira/browse/OAK-4552 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.0, 1.2, 1.4 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > Fix For: 1.6, 1.5.6 > > > The implementation returns a checkpoint based on the expiryTime. Instead it > should only rely on the revision (-vector) of the checkpoint to find the > oldest valid checkpoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4826) Auto removal of orphaned checkpoints
Chetan Mehrotra created OAK-4826:
Summary: Auto removal of orphaned checkpoints
Key: OAK-4826
URL: https://issues.apache.org/jira/browse/OAK-4826
Project: Jackrabbit Oak
Issue Type: Improvement
Components: core
Reporter: Chetan Mehrotra
Fix For: 1.6
Currently if in a running system there are some orphaned checkpoint present
then they prevent the revision gc (compaction for segment) from being
effective.
So far the practice has been to use {{oak-run checkpoints rm-unreferenced}}
command to clean them up manually. This was set to manual as it was not
possible to determine whether current checkpoint is in use or not.
rm-unreferenced works with the basis that checkpoints are only made from
AsyncIndexUpdate and hence can check if the checkpoint is in use by cross
checking with {{:async}} state. Doing it in auto mode is risky as
{{checkpoint}} api can be used by any module.
With OAK-2314 we also record some metadata like {{creator}} and {{name}}. This
can be used for auto cleanup. For example in some running system following
checkpoints are listed
{noformat}
Mon Sep 19 18:02:09 EDT 2016Sun Jun 16 18:02:09 EDT 2019
r15744787d0a-1-1
creator=AsyncIndexUpdate
name=fulltext-async
thread=sling-default-4070-Registered Service.653
Mon Sep 19 18:02:09 EDT 2016Sun Jun 16 18:02:09 EDT 2019
r15744787d0a-0-1
creator=AsyncIndexUpdate
name=async
thread=sling-default-4072-Registered Service.656
Fri Aug 19 18:57:33 EDT 2016Thu May 16 18:57:33 EDT 2019
r156a50612e1-1-1
creator=AsyncIndexUpdate
name=async
thread=sling-default-10-Registered Service.654
Wed Aug 10 12:13:20 EDT 2016Tue May 07 12:25:52 EDT 2019
r156753ac38d-0-1
creator=AsyncIndexUpdate
name=async
thread=sling-default-6041-Registered Service.1966
{noformat}
As can be seen that last 2 checkpoints are orphan and they would prevent
revision gc. For auto mode we can use following heuristic
# List all current checkpoints
# Only keep the latest checkpoint for given {{creator}} and {{name}} combo.
Other entries from same pair which are older i.e. creation time can be consider
orphan and deleted
This logic can be implemented
{{org.apache.jackrabbit.oak.checkpoint.Checkpoints}} and can be invoked by
Revision GC logic (both in DocumentNodeStore and SegmentNodeStore) to determine
the base revision to keep
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
[
https://issues.apache.org/jira/browse/OAK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505286#comment-15505286
]
Tobias Bocanegra commented on OAK-4825:
---
I like the idea:
{noformat}
/**
* Controls the behavior for users that no longer exist on the external
provider. The default is to delete the repository users
* if they no longer exist on the external provider. If set to true, they will
be disabled instead, and re-enabled once they appear
* again.
*/
boolean disableMissingUsers;
{noformat}
we could add a bit more flexibility and add a auto-purge time instead of just a
flag.
eg:
{noformat}
/**
* Timespan (human notation) after which an externally deleted user is purged.
A user that is no longer available on the external
* provider will linger in the repository as a disabled user until this
expiration time has passed, after which it will be deleted.
* Use 0 to delete immediately, -1 to never delete.
*
* Example: "30d" will invalidate a no longer existing user and delete it after
30 days"
*/
String userLingerTime;
{noformat}
> Support disabling of users instead of removal in DefaultSyncHandler
> ---
>
> Key: OAK-4825
> URL: https://issues.apache.org/jira/browse/OAK-4825
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: auth-external
>Reporter: Alexander Klimetschek
>
> The DefaultSyncHandler by default will remove (local) users when they are no
> longer active in the external system aka no longer provided by the
> ExternalIdentityProvider. It would be useful to have an option to _disable_
> users instead of removing them completely.
> This is good for use cases that need to keep the user's data around in the
> JCR and can't just delete it. Also, we have seen cases where the user is only
> temporarily removed from the external identity system (e.g. accidentally
> removed from group that maps them to the JCR system and quickly added back),
> where a full removal can do harm.
> (Note: There is an [option in the SyncContext
> interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38]
> to suppress purging, and the JMX sync commands such as
> [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256]
> "use" it. However, the JCR users look like "valid" users then locally. Even
> if the authentication is done completely through the IDP and will fail
> properly for these missing users, it can be difficult for other uses like
> administration, monitoring, other application code to detect that such a user
> is not active anymore.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
[ https://issues.apache.org/jira/browse/OAK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-4825: --- Description: The DefaultSyncHandler by default will remove (local) users when they are no longer active in the external system aka no longer provided by the ExternalIdentityProvider. It would be useful to have an option to _disable_ users instead of removing them completely. This is good for use cases that need to keep the user's data around in the JCR and can't just delete it. Also, we have seen cases where the user is only temporarily removed from the external identity system (e.g. accidentally removed from group that maps them to the JCR system and quickly added back), where a full removal can do harm. (Note: There is an [option in the SyncContext interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38] to suppress purging, and the JMX sync commands such as [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256] "use" it. However, the JCR users look like "valid" users then locally. Even if the authentication is done completely through the IDP and will fail properly for these missing users, it can be difficult for other uses like administration, monitoring, other application code to detect that such a user is not active anymore.) was: The DefaultSyncHandler by default will remove (local) users when they are no longer active in the external system aka no longer provided by the ExternalIdentityProvider. It would be useful to have an option to _disable_ users instead of removing them completely. This is good for use cases that need to keep the user's data around in the JCR and can't just delete it. Also, we have seen cases where the user is only temporarily removed from the external identity system (e.g. accidentally removed from group that maps them to the JCR system and quickly added back), where a full removal can do harm. (Note: There is an [option in the SyncContext interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38] to suppress purging, and the JMX sync commands such as [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256] "use" it. However, the users look like "valid" users then. Even if the authentication is done completely through the IDP and will fail properly for these missing users, it can be difficult for other uses like administration, monitoring, other application code to detect that such a user is not active anymore.) > Support disabling of users instead of removal in DefaultSyncHandler > --- > > Key: OAK-4825 > URL: https://issues.apache.org/jira/browse/OAK-4825 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: auth-external >Reporter: Alexander Klimetschek > > The DefaultSyncHandler by default will remove (local) users when they are no > longer active in the external system aka no longer provided by the > ExternalIdentityProvider. It would be useful to have an option to _disable_ > users instead of removing them completely. > This is good for use cases that need to keep the user's data around in the > JCR and can't just delete it. Also, we have seen cases where the user is only > temporarily removed from the external identity system (e.g. accidentally > removed from group that maps them to the JCR system and quickly added back), > where a full removal can do harm. > (Note: There is an [option in the SyncContext > interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38] > to suppress purging, and the JMX sync commands such as > [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256] > "use" it. However, the JCR users look like "valid" users then locally. Even > if the authentication is done completely through the IDP and will fail > properly for these missing users, it can be difficult for other uses like > administration, monitoring, other application code to detect that such a user > is not active anymore.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
[ https://issues.apache.org/jira/browse/OAK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Klimetschek updated OAK-4825: --- Description: The DefaultSyncHandler by default will remove (local) users when they are no longer active in the external system aka no longer provided by the ExternalIdentityProvider. It would be useful to have an option to _disable_ users instead of removing them completely. This is good for use cases that need to keep the user's data around in the JCR and can't just delete it. Also, we have seen cases where the user is only temporarily removed from the external identity system (e.g. accidentally removed from group that maps them to the JCR system and quickly added back), where a full removal can do harm. (Note: There is an [option in the SyncContext interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38] to suppress purging, and the JMX sync commands such as [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256] "use" it. However, the users look like "valid" users then. Even if the authentication is done completely through the IDP and will fail properly for these missing users, it can be difficult for other uses like administration, monitoring, other application code to detect that such a user is not active anymore.) was: The DefaultSyncHandler by default will remove of (local) users when they are no longer active in the external system aka no longer provided by the ExternalIdentityProvider. It would be useful to have an option to _disable_ users instead of removing them completely. This is good for use cases that need to keep the user's data around in the JCR and can't just delete it. Also, we have seen cases where the user is only temporarily removed from the external identity system (e.g. accidentally removed from group that maps them to the JCR system and quickly added back), where a full removal can do harm. (Note: There is an [option in the SyncContext interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38] to suppress purging, and the JMX sync commands such as [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256] "use" it. However, the users look like "valid" users then. Even if the authentication is done completely through the IDP and will fail properly for these missing users, it can be difficult for other uses like administration, monitoring, other application code to detect that such a user is not active anymore.) > Support disabling of users instead of removal in DefaultSyncHandler > --- > > Key: OAK-4825 > URL: https://issues.apache.org/jira/browse/OAK-4825 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: auth-external >Reporter: Alexander Klimetschek > > The DefaultSyncHandler by default will remove (local) users when they are no > longer active in the external system aka no longer provided by the > ExternalIdentityProvider. It would be useful to have an option to _disable_ > users instead of removing them completely. > This is good for use cases that need to keep the user's data around in the > JCR and can't just delete it. Also, we have seen cases where the user is only > temporarily removed from the external identity system (e.g. accidentally > removed from group that maps them to the JCR system and quickly added back), > where a full removal can do harm. > (Note: There is an [option in the SyncContext > interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38] > to suppress purging, and the JMX sync commands such as > [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256] > "use" it. However, the users look like "valid" users then. Even if the > authentication is done completely through the IDP and will fail properly for > these missing users, it can be difficult for other uses like administration, > monitoring, other application code to detect that such a user is not active > anymore.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
[
https://issues.apache.org/jira/browse/OAK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504785#comment-15504785
]
Alexander Klimetschek edited comment on OAK-4825 at 9/19/16 9:53 PM:
-
The implementation would have to do something along these lines:
* add configuration option on the DefaultSyncHandler {{disableUsers}} which is
false by default (= removal)
* disable users instead of removing them inside
[DefaultSyncContext.handleMissingIdentity()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/basic/DefaultSyncContext.java#L328-L332]
if {{disableUsers=true}}
* ensure users (and groups) are re-enabled if they come back, do this in
[DefaultSyncContext.syncExternalIdentity()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/basic/DefaultSyncContext.java#L465-L470]
(for both users and groups)
* (maybe something inside the [JMX bean
implementation|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java]
as well, for a consistent behavior regarding purging, not sure)
I am currently doing this in custom code, which is possible, but slightly ugly,
as I have to do the disabling in a custom batch sync code (alternatively copy
the logic from DefaultSyncContext.java completely and overwrite), and the
re-enabling in a custom SyncHandler that returns DefaultSyncContext and only
overwrites syncUser() with the re-enable logic. A custom DefaultSyncHandler
that works mostly the same means that you have to copy the entire
DefaultSyncConfigImpl for the configuration part, which is a bit suboptimal.
was (Author: alexander.klimetschek):
The implementation would have to do something along these lines:
* add configuration option on the DefaultSyncHandler {{disableUsers}} which is
false by default (= removal)
* disable users instead of removing them inside
[DefaultSyncContext.handleMissingIdentity()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/basic/DefaultSyncContext.java#L328-L332]
if {{disableUsers=true}}
* ensure users (and groups) are re-enabled if they come back, do this in
[DefaultSyncContext.syncExternalIdentity()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/basic/DefaultSyncContext.java#L465-L470]
(for both users and groups)
* (maybe something inside the [JMX bean
implementation|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java]
as well, for a consistent behavior regarding purging, not sure)
> Support disabling of users instead of removal in DefaultSyncHandler
> ---
>
> Key: OAK-4825
> URL: https://issues.apache.org/jira/browse/OAK-4825
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: auth-external
>Reporter: Alexander Klimetschek
>
> The DefaultSyncHandler by default will remove of (local) users when they are
> no longer active in the external system aka no longer provided by the
> ExternalIdentityProvider. It would be useful to have an option to _disable_
> users instead of removing them completely.
> This is good for use cases that need to keep the user's data around in the
> JCR and can't just delete it. Also, we have seen cases where the user is only
> temporarily removed from the external identity system (e.g. accidentally
> removed from group that maps them to the JCR system and quickly added back),
> where a full removal can do harm.
> (Note: There is an [option in the SyncContext
> interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38]
> to suppress purging, and the JMX sync commands such as
> [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256]
> "use" it. However, the users look like "valid" users then. Even if the
> authentication is done completely through the IDP and will fail properly for
> these missing users, it can be difficult for other uses like administration,
> monitoring, other application code to detect that such a user is not active
> anymore.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
[
https://issues.apache.org/jira/browse/OAK-4825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504785#comment-15504785
]
Alexander Klimetschek commented on OAK-4825:
The implementation would have to do something along these lines:
* add configuration option on the DefaultSyncHandler {{disableUsers}} which is
false by default (= removal)
* disable users instead of removing them inside
[DefaultSyncContext.handleMissingIdentity()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/basic/DefaultSyncContext.java#L328-L332]
if {{disableUsers=true}}
* ensure users (and groups) are re-enabled if they come back, do this in
[DefaultSyncContext.syncExternalIdentity()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/basic/DefaultSyncContext.java#L465-L470]
(for both users and groups)
* (maybe something inside the [JMX bean
implementation|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java]
as well, for a consistent behavior regarding purging, not sure)
> Support disabling of users instead of removal in DefaultSyncHandler
> ---
>
> Key: OAK-4825
> URL: https://issues.apache.org/jira/browse/OAK-4825
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: auth-external
>Reporter: Alexander Klimetschek
>
> The DefaultSyncHandler by default will remove of (local) users when they are
> no longer active in the external system aka no longer provided by the
> ExternalIdentityProvider. It would be useful to have an option to _disable_
> users instead of removing them completely.
> This is good for use cases that need to keep the user's data around in the
> JCR and can't just delete it. Also, we have seen cases where the user is only
> temporarily removed from the external identity system (e.g. accidentally
> removed from group that maps them to the JCR system and quickly added back),
> where a full removal can do harm.
> (Note: There is an [option in the SyncContext
> interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38]
> to suppress purging, and the JMX sync commands such as
> [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256]
> "use" it. However, the users look like "valid" users then. Even if the
> authentication is done completely through the IDP and will fail properly for
> these missing users, it can be difficult for other uses like administration,
> monitoring, other application code to detect that such a user is not active
> anymore.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4825) Support disabling of users instead of removal in DefaultSyncHandler
Alexander Klimetschek created OAK-4825: -- Summary: Support disabling of users instead of removal in DefaultSyncHandler Key: OAK-4825 URL: https://issues.apache.org/jira/browse/OAK-4825 Project: Jackrabbit Oak Issue Type: Improvement Components: auth-external Reporter: Alexander Klimetschek The DefaultSyncHandler by default will remove of (local) users when they are no longer active in the external system aka no longer provided by the ExternalIdentityProvider. It would be useful to have an option to _disable_ users instead of removing them completely. This is good for use cases that need to keep the user's data around in the JCR and can't just delete it. Also, we have seen cases where the user is only temporarily removed from the external identity system (e.g. accidentally removed from group that maps them to the JCR system and quickly added back), where a full removal can do harm. (Note: There is an [option in the SyncContext interface|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/SyncContext.java#L38] to suppress purging, and the JMX sync commands such as [purgeOrphanedUsers()|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-auth-external/src/main/java/org/apache/jackrabbit/oak/spi/security/authentication/external/impl/jmx/Delegatee.java#L256] "use" it. However, the users look like "valid" users then. Even if the authentication is done completely through the IDP and will fail properly for these missing users, it can be difficult for other uses like administration, monitoring, other application code to detect that such a user is not active anymore.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4824) Deadlock in TarWriter
[
https://issues.apache.org/jira/browse/OAK-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504746#comment-15504746
]
Michael Dürig commented on OAK-4824:
Yes, that should do the trick.
> Deadlock in TarWriter
> -
>
> Key: OAK-4824
> URL: https://issues.apache.org/jira/browse/OAK-4824
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Priority: Critical
> Labels: deadlock, resilience, threading
> Fix For: Segment Tar 0.0.14
>
>
> There is the potential for a deadlock between concurrent calls to
> {{TarWriter#createNextGeneration()}} and {{TarWriter#flush()}}: both methods
> try to acquire a lock on this and another lock on {{TarWriter.file}} but in
> different order. I observed the deadlock when running
> {{CompactionAndCleanupIT.randomAccessFileConcurrentReadAndLength()}}.
> This is a regression introduced with OAK-4746: the method
> {{TarWriter.createNextGeneration()}} seems over eagerly synchronized. I would
> argue that we could drop synchronization for that method entirely as the part
> after the call to {{close()}} will in any case only ever be executed once by
> a single thread. All other threads will fail with an
> {{IllegalStateException}}).
> [~alexparvulescu], WDYT?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4161) DefaultSyncHandler should avoid concurrent synchronization of the same user
[ https://issues.apache.org/jira/browse/OAK-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504719#comment-15504719 ] Alexander Klimetschek commented on OAK-4161: The expiration timeout options of the DefaultSyncHandler might help, or do you see the issue on a "lower" level? > DefaultSyncHandler should avoid concurrent synchronization of the same user > --- > > Key: OAK-4161 > URL: https://issues.apache.org/jira/browse/OAK-4161 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: auth-external >Reporter: Manfred Baedke >Assignee: Manfred Baedke > > Concurrent synchronization of the same user may have a significant > performance impact on systems where user sync is already a bottleneck. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4161) DefaultSyncHandler should avoid concurrent synchronization of the same user
[ https://issues.apache.org/jira/browse/OAK-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504719#comment-15504719 ] Alexander Klimetschek edited comment on OAK-4161 at 9/19/16 9:17 PM: - The expiration timeout options of the DefaultSyncHandler might help (especially "User Expiration Time" IIUC), or do you see the issue on a "lower" level? was (Author: alexander.klimetschek): The expiration timeout options of the DefaultSyncHandler might help, or do you see the issue on a "lower" level? > DefaultSyncHandler should avoid concurrent synchronization of the same user > --- > > Key: OAK-4161 > URL: https://issues.apache.org/jira/browse/OAK-4161 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: auth-external >Reporter: Manfred Baedke >Assignee: Manfred Baedke > > Concurrent synchronization of the same user may have a significant > performance impact on systems where user sync is already a bottleneck. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4824) Deadlock in TarWriter
[
https://issues.apache.org/jira/browse/OAK-4824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504275#comment-15504275
]
Alex Parvulescu commented on OAK-4824:
--
bq. This is a regression introduced with OAK-4746: the method
TarWriter.createNextGeneration() seems over eagerly synchronized.
right. looks bad. the issue is with accessing the {{access}} var which seems to
always be protected by a lock on the {{TarWriter}} itself. should we wrap only
the {{if (access == null)}} check in a {{synchronized (this)}} block?
> Deadlock in TarWriter
> -
>
> Key: OAK-4824
> URL: https://issues.apache.org/jira/browse/OAK-4824
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Priority: Critical
> Labels: deadlock, resilience, threading
> Fix For: Segment Tar 0.0.14
>
>
> There is the potential for a deadlock between concurrent calls to
> {{TarWriter#createNextGeneration()}} and {{TarWriter#flush()}}: both methods
> try to acquire a lock on this and another lock on {{TarWriter.file}} but in
> different order. I observed the deadlock when running
> {{CompactionAndCleanupIT.randomAccessFileConcurrentReadAndLength()}}.
> This is a regression introduced with OAK-4746: the method
> {{TarWriter.createNextGeneration()}} seems over eagerly synchronized. I would
> argue that we could drop synchronization for that method entirely as the part
> after the call to {{close()}} will in any case only ever be executed once by
> a single thread. All other threads will fail with an
> {{IllegalStateException}}).
> [~alexparvulescu], WDYT?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4824) Deadlock in TarWriter
Michael Dürig created OAK-4824:
--
Summary: Deadlock in TarWriter
Key: OAK-4824
URL: https://issues.apache.org/jira/browse/OAK-4824
Project: Jackrabbit Oak
Issue Type: Bug
Components: segment-tar
Reporter: Michael Dürig
Priority: Critical
Fix For: Segment Tar 0.0.14
There is the potential for a deadlock between concurrent calls to
{{TarWriter#createNextGeneration()}} and {{TarWriter#flush()}}: both methods
try to acquire a lock on this and another lock on {{TarWriter.file}} but in
different order. I observed the deadlock when running
{{CompactionAndCleanupIT.randomAccessFileConcurrentReadAndLength()}}.
This is a regression introduced with OAK-4746: the method
{{TarWriter.createNextGeneration()}} seems over eagerly synchronized. I would
argue that we could drop synchronization for that method entirely as the part
after the call to {{close()}} will in any case only ever be executed once by a
single thread. All other threads will fail with an {{IllegalStateException}}).
[~alexparvulescu], WDYT?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4283) Align GCMonitor API with implementation
[
https://issues.apache.org/jira/browse/OAK-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4283:
---
Fix Version/s: (was: Segment Tar 0.0.14)
Segment Tar 0.0.16
> Align GCMonitor API with implementation
>
>
> Key: OAK-4283
> URL: https://issues.apache.org/jira/browse/OAK-4283
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: api-change, compaction, gc
> Fix For: Segment Tar 0.0.16
>
>
> The argument taken by {{GCMonitor.compacted}} related to parameters of the
> compaction map. The latter has gone with OAK-3348. We need to come up with a
> way to adjust this API accordingly. Also it might make sense to broaden the
> scope of {{GCMonitor}} from its initial intent (logging) to a more general
> one as this is how it is already used e.g. by the {{RefreshOnGC}}
> implementation and for OAK-4096.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4617) Align SegmentRevisionGC MBean with new generation based GC
[
https://issues.apache.org/jira/browse/OAK-4617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4617:
---
Fix Version/s: (was: Segment Tar 0.0.14)
Segment Tar 0.0.16
> Align SegmentRevisionGC MBean with new generation based GC
> --
>
> Key: OAK-4617
> URL: https://issues.apache.org/jira/browse/OAK-4617
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: production
> Fix For: Segment Tar 0.0.16
>
>
> The {{SegmentRevisionGC}} MBean still dates back to the old {{oak-segment}}.
> We need to review its endpoints and only keep those that make sense for
> {{oak-segment-tar}}, adapt the others as necessary any add further
> functionality as required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4742) Improve FileStoreStatsMBean
[ https://issues.apache.org/jira/browse/OAK-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4742: --- Fix Version/s: (was: Segment Tar 0.0.14) Segment Tar 0.0.16 > Improve FileStoreStatsMBean > --- > > Key: OAK-4742 > URL: https://issues.apache.org/jira/browse/OAK-4742 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Michael Dürig > Labels: monitoring > Fix For: Segment Tar 0.0.16 > > > We should add further data to that MBean (if feasible): > * Number of commits > * Number of commits queuing (blocked on the commit semaphore) > * Percentiles of commit times (exclude queueing time) > * Percentiles of commit queueing times -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3695) Expose ratio between waste and real data
[ https://issues.apache.org/jira/browse/OAK-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-3695: --- Fix Version/s: (was: Segment Tar 0.0.14) Segment Tar 0.0.16 > Expose ratio between waste and real data > > > Key: OAK-3695 > URL: https://issues.apache.org/jira/browse/OAK-3695 > Project: Jackrabbit Oak > Issue Type: Story > Components: segment-tar >Reporter: Valentin Olteanu > Labels: gc > Fix For: Segment Tar 0.0.16 > > > As a user, I want to know the ratio (or precise absolute values) between > waste and real data on TarMK, so that I can decide if Revision GC needs to be > run. The measurement has to be done on a running repository and without > impacting the performance. > This would also help measure the efficiency of Revision GC and see the effect > of improvements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4685) Avoid concurrent calls to FileStore.cleanup() and FileStore.compact()
[
https://issues.apache.org/jira/browse/OAK-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4685:
---
Fix Version/s: (was: Segment Tar 0.0.12)
Segment Tar 0.0.14
> Avoid concurrent calls to FileStore.cleanup() and FileStore.compact()
> -
>
> Key: OAK-4685
> URL: https://issues.apache.org/jira/browse/OAK-4685
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar, segmentmk
>Affects Versions: 1.0.32, 1.4.6, 1.2.18, Segment Tar 0.0.8
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
> Labels: cleanup, gc
> Fix For: Segment Tar 0.0.14
>
>
> At the moment it is possible to have concurrent calls to
> {{FileStore.cleanup}} and to {{FileStore.compact()}}. The former is called
> from the latter and also from {{FileStore.flush()}} (this is tracked in
> OAK-4138). We should change this status quo and also make the calls to
> {{compact()}} and {{cleanup()}} mutually exclusive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4765) Provide option to interrupt online revision cleanup
[ https://issues.apache.org/jira/browse/OAK-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4765: --- Fix Version/s: (was: Segment Tar 0.0.12) Segment Tar 0.0.14 > Provide option to interrupt online revision cleanup > --- > > Key: OAK-4765 > URL: https://issues.apache.org/jira/browse/OAK-4765 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar, segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: compaction, gc > Fix For: Segment Tar 0.0.14 > > > JMX binding for stopping a running compaction process -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4659) Address records by logic identifiers instead of offsets
[ https://issues.apache.org/jira/browse/OAK-4659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4659: --- Fix Version/s: (was: Segment Tar 0.0.12) Segment Tar 0.0.14 > Address records by logic identifiers instead of offsets > --- > > Key: OAK-4659 > URL: https://issues.apache.org/jira/browse/OAK-4659 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: segment-tar >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: Segment Tar 0.0.14 > > Attachments: OAK-4659-01.patch, OAK-4659-02.patch > > > Records are currently identified by their offsets inside the segments, This > prevents records from being relocatable inside a segment and, as a > consequence, this prevents segments to be shrunk down. This limitation forced > us into a very limited solution space when working on garbage collection. > Logic record IDs and their impact on the current implementation should be > investigated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4813) Simplify the server side of cold standby
[
https://issues.apache.org/jira/browse/OAK-4813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4813:
---
Fix Version/s: (was: Segment Tar 0.0.12)
Segment Tar 0.0.14
> Simplify the server side of cold standby
>
>
> Key: OAK-4813
> URL: https://issues.apache.org/jira/browse/OAK-4813
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Andrei Dulceanu
>Assignee: Andrei Dulceanu
>Priority: Minor
> Fix For: Segment Tar 0.0.14
>
>
> With the changes introduced in OAK-4803, it would be nice to keep the
> previous symmetry between the client and server and remove thus the
> {{FileStore}} reference from the latter.
> Per [~frm]'s suggestion from one of the comments in OAK-4803:
> bq. In the end, these are the only three lines where the FileStore is used in
> the server, which already suggests that this separation of concerns exists -
> at least at the level of the handlers.
> {code:java}
> p.addLast(new GetHeadRequestHandler(new DefaultStandbyHeadReader(store)));
> p.addLast(new GetSegmentRequestHandler(new
> DefaultStandbySegmentReader(store)));
> p.addLast(new GetBlobRequestHandler(new DefaultStandbyBlobReader(store)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4822) Upgrade Oak dependency to 1.5.10
[ https://issues.apache.org/jira/browse/OAK-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-4822. Resolution: Fixed Fixed at http://svn.apache.org/viewvc?rev=1761472=rev > Upgrade Oak dependency to 1.5.10 > > > Key: OAK-4822 > URL: https://issues.apache.org/jira/browse/OAK-4822 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig > Fix For: Segment Tar 0.0.14 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4822) Upgrade Oak dependency to 1.5.10
[ https://issues.apache.org/jira/browse/OAK-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4822: --- Summary: Upgrade Oak dependency to 1.5.10 (was: Upgrade Oak dependency to 1.5.9) > Upgrade Oak dependency to 1.5.10 > > > Key: OAK-4822 > URL: https://issues.apache.org/jira/browse/OAK-4822 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig > Fix For: Segment Tar 0.0.14 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4823) Upgrade Oak Segment Tar dependency to 0.0.12
[ https://issues.apache.org/jira/browse/OAK-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-4823. Resolution: Fixed Fixed at http://svn.apache.org/viewvc?rev=1761467=rev > Upgrade Oak Segment Tar dependency to 0.0.12 > > > Key: OAK-4823 > URL: https://issues.apache.org/jira/browse/OAK-4823 > Project: Jackrabbit Oak > Issue Type: Task > Components: parent >Reporter: Michael Dürig >Assignee: Michael Dürig > Fix For: 1.5.11 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4823) Upgrade Oak Segment Tar dependency to 0.0.12
Michael Dürig created OAK-4823: -- Summary: Upgrade Oak Segment Tar dependency to 0.0.12 Key: OAK-4823 URL: https://issues.apache.org/jira/browse/OAK-4823 Project: Jackrabbit Oak Issue Type: Task Components: parent Reporter: Michael Dürig Assignee: Michael Dürig Fix For: 1.5.11 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4822) Upgrade Oak dependency to 1.5.9
[ https://issues.apache.org/jira/browse/OAK-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4822: --- Fix Version/s: Segment Tar 0.0.14 > Upgrade Oak dependency to 1.5.9 > --- > > Key: OAK-4822 > URL: https://issues.apache.org/jira/browse/OAK-4822 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig > Fix For: Segment Tar 0.0.14 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4822) Upgrade Oak dependency to 1.5.9
Michael Dürig created OAK-4822: -- Summary: Upgrade Oak dependency to 1.5.9 Key: OAK-4822 URL: https://issues.apache.org/jira/browse/OAK-4822 Project: Jackrabbit Oak Issue Type: Task Components: segment-tar Reporter: Michael Dürig Assignee: Michael Dürig -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503621#comment-15503621
]
Chetan Mehrotra edited comment on OAK-4796 at 9/19/16 2:23 PM:
---
What a coincidence was just looking at this issue and now the patch!. Have not
yet gone in detail into the patch but wanted to comment on approach. Instead of
having a Validator provided by Observer determine if certain change is
interesting for given observor we can take a more decouple approach
h3. A - Collect data
For any given commit collect set of potentially interesting things (as
mentioned in OAK-4586)
# set of nodetype names for modified nodes
# set of property names which got modified
# Paths (upto n level) under which modification happened say upto depth 2
# set of node names which got modified
This logic can be implemented as an {{Editor}} which gets registered in
{{IndexUpdate}} and it collects above data and stores it in {{CommitContext}}
on _best effort basis_. The editor would be invoked for each change and there
it can very efficiently build up this state. Complete data is collected
irrespective if any observer is interested in that change.
My understanding is that this state would not be very large for most of the
commits done
By best effort means that if any data becomes too big say 1000 different
property name (highly unlikely!) which got changed then it would empty the
state and somehow indicate that data is too large and observer should do the
hard work. So if we are in a large transaction then we do not collect this data
(configurable limits)
h3. B - Filter out based on collected data
Each observer would then provide a Filter which would be run when any
ContentChange gets enqueued and it only allows those changes which have changes
which it is interested in
This approach would allow us to later serialized this collected data in
DocumentNodeStore journal entry and gets merged when an external diff event is
sent. Thus benefiting both local and external change processing.
was (Author: chetanm):
What a coincidence was just looking at this issue and now the patch!. Have not
yet gone in detail into the patch but wanted to commit on approach. Instead of
having a Validator provided by Observer determine if certain change is
interesting for given observor we can take a more decouple approach
h3. A - Collect data
For any given commit collect set of potentially interesting things (as
mentioned in OAK-4586)
# set of nodetype names for modified nodes
# set of property names which got modified
# Paths (upto n level) under which modification happened say upto depth 2
# set of node names which got modified
This logic can be implemented as an {{Editor}} which gets registered in
{{IndexUpdate}} and it collects above data and stores it in {{CommitContext}}
on _best effort basis_. The editor would be invoked for each change and there
it can very efficiently build up this state. Complete data is collected
irrespective if any observer is interested in that change.
My understanding is that this state would not be very large for most of the
commits done
By best effort means that if any data becomes too big say 1000 different
property name which got changed then it would not empty the state and indicate
that data is too large and observer should do the hard work. So if we are in a
large commit then we do not collect this data (configurable limits)
h3. B - Filter out based on collected data
Each observer would then provide a Filter which would be run when any
ContentChange gets enqueued and it only allows those changes which have changes
which it is interested in
This approach would allow us to later serialized this collected data in
DocumentNodeStore journal entry and gets merged when an external diff event is
sent. Thus benefiting both local and external change processing.
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> #
[jira] [Commented] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503621#comment-15503621
]
Chetan Mehrotra commented on OAK-4796:
--
What a coincidence was just looking at this issue and now the patch!. Have not
yet gone in detail into the patch but wanted to commit on approach. Instead of
having a Validator provided by Observer determine if certain change is
interesting for given observor we can take a more decouple approach
h3. A - Collect data
For any given commit collect set of potentially interesting things (as
mentioned in OAK-4586)
# set of nodetype names for modified nodes
# set of property names which got modified
# Paths (upto n level) under which modification happened say upto depth 2
# set of node names which got modified
This logic can be implemented as an {{Editor}} which gets registered in
{{IndexUpdate}} and it collects above data and stores it in {{CommitContext}}
on _best effort basis_. The editor would be invoked for each change and there
it can very efficiently build up this state. Complete data is collected
irrespective if any observer is interested in that change.
My understanding is that this state would not be very large for most of the
commits done
By best effort means that if any data becomes too big say 1000 different
property name which got changed then it would not empty the state and indicate
that data is too large and observer should do the hard work. So if we are in a
large commit then we do not collect this data (configurable limits)
h3. B - Filter out based on collected data
Each observer would then provide a Filter which would be run when any
ContentChange gets enqueued and it only allows those changes which have changes
which it is interested in
This approach would allow us to later serialized this collected data in
DocumentNodeStore journal entry and gets merged when an external diff event is
sent. Thus benefiting both local and external change processing.
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> # events (in the form of ContentChange Objects) occupy a slot of the queue
> even if the listener is not interested in it - any commit lands on any
> listener's queue. This reduces the capacity of the queue for 'actual' events
> to be delivered. It therefore increases the risk that the queue fills - and
> when full has various consequences such as loosing the CommitInfo etc.
> # each event==ContentChange later on must be evaluated, and for that a diff
> must be calculated. Depending on runtime behavior that diff might be
> expensive if no longer in the cache (documentMk specifically).
> As an improvement, this diffing+filtering could be done at an earlier stage
> already, nearer to the commit, and in case the filter would ignore the event,
> it would not have to be put into the queue at all, thus avoiding occupying a
> slot and later potentially slower diffing.
> The suggestion is to implement this via the following algorithm:
> * During the commit, in a {{Validator}} the listener's filters are evaluated
> - in an as-efficient-as-possible manner (Reason for doing it in a Validator
> is that this doesn't add overhead as oak already goes through all changes for
> other Validators). As a result a _list of potentially affected observers_ is
> added to the {{CommitInfo}} (false positives are fine).
> ** Note that the above adds cost to the commit and must therefore be
> carefully done and measured
> ** One potential measure could be to only do filtering when listener's queues
> are larger than a certain threshold (eg 10)
> * The ChangeProcessor in {{contentChanged}} (in the one created in
> [createObserver|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L224])
> then checks the new commitInfo's _potentially affected observers_ list and
> if it's not in the
[jira] [Updated] (OAK-4819) Improve revision GC resilience
[ https://issues.apache.org/jira/browse/OAK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-4819: -- Labels: candidate_oak_1_0 candidate_oak_1_2 candidate_oak_1_4 resilience (was: ) > Improve revision GC resilience > -- > > Key: OAK-4819 > URL: https://issues.apache.org/jira/browse/OAK-4819 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4, > resilience > Fix For: 1.6, 1.5.11 > > > Revision GC may currently fail when a document with a malformed id is read > from the DocumentStore. E.g. a document stored accidentally in the nodes > collection or malformed for some other reason. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4819) Improve revision GC resilience
[ https://issues.apache.org/jira/browse/OAK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-4819. --- Resolution: Fixed Fix Version/s: 1.5.11 Revision GC now checks document IDs and skips/logs those documents with invalid IDs. Implemented in trunk: http://svn.apache.org/r1761444 > Improve revision GC resilience > -- > > Key: OAK-4819 > URL: https://issues.apache.org/jira/browse/OAK-4819 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.6, 1.5.11 > > > Revision GC may currently fail when a document with a malformed id is read > from the DocumentStore. E.g. a document stored accidentally in the nodes > collection or malformed for some other reason. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Attachment: OAK-4583.diff tested, but Tomcat release not officially announced yet > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > Attachments: OAK-4583.diff > > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4796) filter events before adding to ChangeProcessor's queue
[
https://issues.apache.org/jira/browse/OAK-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-4796:
-
Attachment: OAK-4796.patch
Attaching patch ([^OAK-4796.patch]) which contains the functionality as
described, here's the implementation again in detail:
* An Observer can now voluntarily implement an extension interface
{{ObserverValidatorProvider}} which has a {{Validator getRootValidator()}}
method
* The new ObservationFilterValidatorProvider is the main integrator: it has a
reference to all such ObserverValidatorProviders and hooks them into the
validators via a CompositeValidator
* There's now an extension to BackgroundObserver called
{{PrefilteringBackgroundObserver}} which implements the new
ObserverValidatorProvider and does so by mapping the Validator interface to the
EventFilter interface.
* The BackgroundObserver handles the new
{{"oak.observation.observerFiltersEvaluated"}} and
{{"oak.observation.interestedObservers"}} properties that are set on the
CommitContext and if it figures a listener does *not* need any event for a
commit, marks the *next non filtered* ContentChange with a new
{{noopPreviousRoot}} (which is a pointer to the last filtered root)
** This noopPreviousRoot is then used to send out a new
{{CommitInfo.NOOP_CHANGE}} token, which indicates to Observers that they should
ignore this contentChanged call (but update the root/previousRoot accordingly).
* oak-jcr's ChangeProcessor now uses such a PrefilteringBackgroundObserver and
does the last puzzle of filtering: it filters the entire change via
{{includeCommit}} (which applies things like isExternal/isInternal)
* to limit potential performance effects the ChangeProcessor has a flag that
control the size of the queue after which prefiltering will be done. Default 20.
Additionally, the patch can be used in 'test' mode, in which case a flag is set
but BackgroundObserver doesn't evaluate it - instead the ChangeProcessor checks
if the flag would have been correct. This test mode would be removed after
enough confidence, but could be used in IT testing to verify that filtering
would have been done correctly.
Pending tasks:
* more test cases, IT
* performance testing
I would appreciate feedback/review of those having been involved in
observation. /cc [~mduerig], [~chetanm], [~catholicon], [~mreutegg]
> filter events before adding to ChangeProcessor's queue
> --
>
> Key: OAK-4796
> URL: https://issues.apache.org/jira/browse/OAK-4796
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Affects Versions: 1.5.9
>Reporter: Stefan Egli
>Assignee: Stefan Egli
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4796.patch
>
>
> Currently the
> [ChangeProcessor.contentChanged|https://github.com/apache/jackrabbit-oak/blob/f4f4e01dd8f708801883260481d37fdcd5868deb/oak-jcr/src/main/java/org/apache/jackrabbit/oak/jcr/observation/ChangeProcessor.java#L335]
> is in charge of doing the event diffing and filtering and does so in a
> pooled Thread, ie asynchronously, at a later stage independent from the
> commit. This has the advantage that the commit is fast, but has the following
> potentially negative effects:
> # events (in the form of ContentChange Objects) occupy a slot of the queue
> even if the listener is not interested in it - any commit lands on any
> listener's queue. This reduces the capacity of the queue for 'actual' events
> to be delivered. It therefore increases the risk that the queue fills - and
> when full has various consequences such as loosing the CommitInfo etc.
> # each event==ContentChange later on must be evaluated, and for that a diff
> must be calculated. Depending on runtime behavior that diff might be
> expensive if no longer in the cache (documentMk specifically).
> As an improvement, this diffing+filtering could be done at an earlier stage
> already, nearer to the commit, and in case the filter would ignore the event,
> it would not have to be put into the queue at all, thus avoiding occupying a
> slot and later potentially slower diffing.
> The suggestion is to implement this via the following algorithm:
> * During the commit, in a {{Validator}} the listener's filters are evaluated
> - in an as-efficient-as-possible manner (Reason for doing it in a Validator
> is that this doesn't add overhead as oak already goes through all changes for
> other Validators). As a result a _list of potentially affected observers_ is
> added to the {{CommitInfo}} (false positives are fine).
> ** Note that the above adds cost to the commit and must therefore be
> carefully done and measured
> ** One potential measure could be to only do filtering when listener's queues
> are larger than a certain threshold (eg 10)
[jira] [Commented] (OAK-4586) Collect affected node types on commit
[ https://issues.apache.org/jira/browse/OAK-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503565#comment-15503565 ] Chetan Mehrotra commented on OAK-4586: -- Another thing to collect is node names whose properties got modified (changed/added/deleted node). This would be useful for certain types of [listeners|http://markmail.org/thread/diixjz6eoipilwf7] /cc [~cziegeler] > Collect affected node types on commit > - > > Key: OAK-4586 > URL: https://issues.apache.org/jira/browse/OAK-4586 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: jcr >Reporter: Marcel Reutegger > Labels: observation > > One of the major optimizations for generating events in oak-jcr is to avoid > traversal into sub trees when there are path filters on an observation > listener. Another optimization could be done based on the node type filters > provided by a listener. If oak-jcr knows the affected node types of a cache > set it could shortcut event processing in case the listeners node type filter > is disjoint. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4820) oak-auth-external can't handle full-width logins
[ https://issues.apache.org/jira/browse/OAK-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manfred Baedke updated OAK-4820: Component/s: security core > oak-auth-external can't handle full-width logins > > > Key: OAK-4820 > URL: https://issues.apache.org/jira/browse/OAK-4820 > Project: Jackrabbit Oak > Issue Type: Bug > Components: auth-external, core, security >Affects Versions: 1.0.33, 1.4.7, 1.2.19 >Reporter: Manfred Baedke >Assignee: Manfred Baedke >Priority: Minor > > oak-auth-external should be optionally be able to conform to > https://tools.ietf.org/html/rfc7613, since the external authentication server > might do so (seen in the wild with LDAP auth against Active Directory). > For Oak this means that the AuthorizableBaseProvider should be able to > generate identical content IDs for strings that are equivalent according to > the UsernameCaseMapped Profile defined in > https://tools.ietf.org/html/rfc7613#section-3.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4820) oak-auth-external can't handle full-width logins
[ https://issues.apache.org/jira/browse/OAK-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manfred Baedke updated OAK-4820: Affects Version/s: (was: 1.4.7) > oak-auth-external can't handle full-width logins > > > Key: OAK-4820 > URL: https://issues.apache.org/jira/browse/OAK-4820 > Project: Jackrabbit Oak > Issue Type: Bug > Components: auth-external >Affects Versions: 1.0.33, 1.4.7, 1.2.19 >Reporter: Manfred Baedke >Assignee: Manfred Baedke >Priority: Minor > > oak-auth-external should be optionally be able to conform to > https://tools.ietf.org/html/rfc7613, since the external authentication server > might do so (seen in the wild with LDAP auth against Active Directory). > For Oak this means that the AuthorizableBaseProvider should be able to > generate identical content IDs for strings that are equivalent according to > the UsernameCaseMapped Profile defined in > https://tools.ietf.org/html/rfc7613#section-3.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4820) oak-auth-external can't handle full-width logins
[ https://issues.apache.org/jira/browse/OAK-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manfred Baedke updated OAK-4820: Affects Version/s: 1.4.7 > oak-auth-external can't handle full-width logins > > > Key: OAK-4820 > URL: https://issues.apache.org/jira/browse/OAK-4820 > Project: Jackrabbit Oak > Issue Type: Bug > Components: auth-external >Affects Versions: 1.0.33, 1.4.7, 1.2.19 >Reporter: Manfred Baedke >Assignee: Manfred Baedke >Priority: Minor > > oak-auth-external should be optionally be able to conform to > https://tools.ietf.org/html/rfc7613, since the external authentication server > might do so (seen in the wild with LDAP auth against Active Directory). > For Oak this means that the AuthorizableBaseProvider should be able to > generate identical content IDs for strings that are equivalent according to > the UsernameCaseMapped Profile defined in > https://tools.ietf.org/html/rfc7613#section-3.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4821) Allow use of Java 7 in Oak 1.4
Julian Reschke created OAK-4821: --- Summary: Allow use of Java 7 in Oak 1.4 Key: OAK-4821 URL: https://issues.apache.org/jira/browse/OAK-4821 Project: Jackrabbit Oak Issue Type: Task Components: parent Affects Versions: 1.4.7 Reporter: Julian Reschke Assignee: Julian Reschke (mailing list discussion in progress) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4820) oak-auth-external can't handle full-width logins
Manfred Baedke created OAK-4820: --- Summary: oak-auth-external can't handle full-width logins Key: OAK-4820 URL: https://issues.apache.org/jira/browse/OAK-4820 Project: Jackrabbit Oak Issue Type: Bug Components: auth-external Affects Versions: 1.2.19, 1.4.7, 1.0.33 Reporter: Manfred Baedke Assignee: Manfred Baedke Priority: Minor oak-auth-external should be optionally be able to conform to https://tools.ietf.org/html/rfc7613, since the external authentication server might do so (seen in the wild with LDAP auth against Active Directory). For Oak this means that the AuthorizableBaseProvider should be able to generate identical content IDs for strings that are equivalent according to the UsernameCaseMapped Profile defined in https://tools.ietf.org/html/rfc7613#section-3.2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4819) Improve revision GC resilience
[ https://issues.apache.org/jira/browse/OAK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503099#comment-15503099 ] Marcel Reutegger commented on OAK-4819: --- See also OAK-3395 for a similar issue. > Improve revision GC resilience > -- > > Key: OAK-4819 > URL: https://issues.apache.org/jira/browse/OAK-4819 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.6 > > > Revision GC may currently fail when a document with a malformed id is read > from the DocumentStore. E.g. a document stored accidentally in the nodes > collection or malformed for some other reason. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4819) Improve revision GC resilience
Marcel Reutegger created OAK-4819: - Summary: Improve revision GC resilience Key: OAK-4819 URL: https://issues.apache.org/jira/browse/OAK-4819 Project: Jackrabbit Oak Issue Type: Improvement Components: core, documentmk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Fix For: 1.6 Revision GC may currently fail when a document with a malformed id is read from the DocumentStore. E.g. a document stored accidentally in the nodes collection or malformed for some other reason. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1312) Bundle nodes into a document
[ https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503050#comment-15503050 ] Marcel Reutegger commented on OAK-1312: --- One more thing, already mentioned to Chetan offline. The MongoDocumentStore could leverage sub document structure when persisting map entries with relative paths. That way many duplicate path prefixes could be avoided. It would be MongoDocumentStore specific and transparent to the DocumentNodeStore. The UpdateOp and also the in-memory Document representation would still have the full relative paths for bundled nodes/properties. In any case, this enhancement should be discussed first in a separate issue. > Bundle nodes into a document > > > Key: OAK-1312 > URL: https://issues.apache.org/jira/browse/OAK-1312 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Chetan Mehrotra > Labels: performance > Fix For: 1.6 > > Attachments: OAK-1312-meta-prop-handling.patch, > OAK-1312-review-v1.diff, OAK-1312-review-v2.diff > > > For very fine grained content with many nodes and only few properties per > node it would be more efficient to bundle multiple nodes into a single > MongoDB document. Mostly reading would benefit because there are less > roundtrips to the backend. At the same time storage footprint would be lower > because metadata overhead is per document. > Feature branch - > https://github.com/chetanmeh/jackrabbit-oak/compare/trunk...chetanmeh:OAK-1312 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Affects Version/s: 1.0.33 1.4.7 1.2.19 1.5.10 > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4583) RDB*Store: update Tomcat JDBC pool dependency
[ https://issues.apache.org/jira/browse/OAK-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4583: Labels: candidate_oak_1_0 candidate_oak_1_2 candidate_oak_1_4 (was: ) > RDB*Store: update Tomcat JDBC pool dependency > - > > Key: OAK-4583 > URL: https://issues.apache.org/jira/browse/OAK-4583 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.4.7, 1.2.19, 1.5.10 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > > ...once 7.0.71 is released (see > https://bz.apache.org/bugzilla/show_bug.cgi?id=59850 and > https://issues.apache.org/jira/browse/OAK-4559) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4733) Reduce DocumentStore reads for local changes (2)
[ https://issues.apache.org/jira/browse/OAK-4733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4733. - bulk close for 1.5.10 > Reduce DocumentStore reads for local changes (2) > > > Key: OAK-4733 > URL: https://issues.apache.org/jira/browse/OAK-4733 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Labels: performance > Fix For: 1.6, 1.5.10 > > > There is another case where the local cache is incorrectly updated, which > leads to unnecessary reads from the DocumentStore. See also OAK-4715. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4792) Replace usage of AssertionError in ClusterNodeInfo
[ https://issues.apache.org/jira/browse/OAK-4792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4792. - bulk close for 1.5.10 > Replace usage of AssertionError in ClusterNodeInfo > -- > > Key: OAK-4792 > URL: https://issues.apache.org/jira/browse/OAK-4792 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.6, 1.5.10 > > > As discussed in OAK-4770, I would like to replace usage of AssertionError in > ClusterNodeInfo with DocumentStoreException and clarify the contract > accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4753) Add nrt as another index mode
[
https://issues.apache.org/jira/browse/OAK-4753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4753.
-
bulk close for 1.5.10
> Add nrt as another index mode
> -
>
> Key: OAK-4753
> URL: https://issues.apache.org/jira/browse/OAK-4753
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: core
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.6, 1.5.10
>
>
> OAK-4641 added "sync" as a new async value to allow same editor to
> participate in async and sync indexing. Similar to that I would like to add
> "nrt" as another mode for async
> So following async value would be supported
> # {{sync}} - The index editor would be invoked as part of sync indexing
> # {{nrt}} - Stands for _near real time_. The index editor would again be
> invoked as part of sync commit
> # {{async}} or any other value - The index editor would be invoked as part of
> matching async indexer run
> In Lucene case LuceneIndexEditor would use "nrt" for those indexes which are
> locally managed but asynchronously updated while "sync" would be used for
> those Lucene indexes which are synchronously updated upon commit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4761) Reduce the number of calls to /jcr:system/jcr:versionStorage during the upgrade
[ https://issues.apache.org/jira/browse/OAK-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4761. - bulk close for 1.5.10 > Reduce the number of calls to /jcr:system/jcr:versionStorage during the > upgrade > --- > > Key: OAK-4761 > URL: https://issues.apache.org/jira/browse/OAK-4761 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: upgrade >Reporter: Tomek Rękawek >Assignee: Tomek Rękawek >Priority: Minor > Fix For: 1.6, 1.5.10 > > > Each time a version history is copied during the upgrade, the copier tool has > to traverse /jcr:system/jcr:versionStorage ancestors to get the appropriate > nt:versionHistory node. It can be optimized by reading the versionStorage > node once and re-using it. As a result, all methods related to copying > versions should accept versionStorage rather than rootNode as their > parameters. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4767) Provide a way to report additional stats as part of benchmark run
[
https://issues.apache.org/jira/browse/OAK-4767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4767.
-
bulk close for 1.5.10
> Provide a way to report additional stats as part of benchmark run
> -
>
> Key: OAK-4767
> URL: https://issues.apache.org/jira/browse/OAK-4767
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.6, 1.5.10
>
>
> Some benchmark also compute additions stats. It would be useful to have them
> reported as part of standard report
> {noformat}
> Apache Jackrabbit Oak
> # HybridIndexTest C min 10% 50% 90% max
> N Searcher Mutator Indexed
> Oak-Segment-Tar-FDS1 70 85 119 259 469
> 34 34563 348 318 #property,numIdxs:50,async:200
> {noformat}
> Following extension are to be provided
> # Add extra column names
> # Add column values along with format string
> # Add an optional comment
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4799) Unnecessary unboxing in ClusterNodeInfo
[ https://issues.apache.org/jira/browse/OAK-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4799. - bulk close for 1.5.10 > Unnecessary unboxing in ClusterNodeInfo > --- > > Key: OAK-4799 > URL: https://issues.apache.org/jira/browse/OAK-4799 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.6, 1.5.10 > > > ClusterNodeInfo unnecessarily unboxes some numeric values. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4674) Log a message when asynchronous persistent cache is enabled
[
https://issues.apache.org/jira/browse/OAK-4674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4674.
-
bulk close for 1.5.10
> Log a message when asynchronous persistent cache is enabled
> ---
>
> Key: OAK-4674
> URL: https://issues.apache.org/jira/browse/OAK-4674
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: cache, documentmk
>Reporter: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6, 1.5.10
>
>
> It would be useful to log a message indicating that asynchronous persistent
> cache (implemented in OAK-2761) is enabled by {{oak.cache.asynchronous}}
> property (introduced in OAK-4123).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4762) Rebuild indexes only for partial sidegrades
[
https://issues.apache.org/jira/browse/OAK-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4762.
-
bulk close for 1.5.10
> Rebuild indexes only for partial sidegrades
> ---
>
> Key: OAK-4762
> URL: https://issues.apache.org/jira/browse/OAK-4762
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.10
>
>
> When doing a sidegrade, the counter and uuid indexes are fully rebuild. This
> should happen only if user specifies custom
> {{\-\-\{include,exclude,merge\}\-path}} list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4705) Fulltext parser doesn't allow stand-alone hyphen in search expression
[
https://issues.apache.org/jira/browse/OAK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4705.
-
bulk close for 1.5.10
> Fulltext parser doesn't allow stand-alone hyphen in search expression
> -
>
> Key: OAK-4705
> URL: https://issues.apache.org/jira/browse/OAK-4705
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
>Priority: Minor
> Fix For: 1.5.10
>
>
> Afaics, the fulltext grammar
> [spec|https://docs.adobe.com/content/docs/en/spec/jcr/2.0/6_Query.html#6.7.19%20FullTextSearch]
> \[0] allows a stand-alone hyphen to be a simple word.
> So, it seems all of these are valid:
> # {{A - B}} //hyphen being a separate word
> # {{A -}} //hyphen being a separate word at the end
> # {{A -B}} //hyphen working as negation operator on {{B}}
> # {{A -"B C"}} //hyphen working as negation operator on {{"B C"}}
> Currently, case 1 and 2 are treated as invalid.
> \[0]:
> https://docs.adobe.com/content/docs/en/spec/jcr/2.0/6_Query.html#6.7.19%20FullTextSearch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4749) Include initial cost in stats for observation processing
[
https://issues.apache.org/jira/browse/OAK-4749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4749.
-
bulk close for 1.5.10
> Include initial cost in stats for observation processing
>
>
> Key: OAK-4749
> URL: https://issues.apache.org/jira/browse/OAK-4749
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: jcr
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
> Labels: observation
> Fix For: 1.6, 1.5.10
>
> Attachments: OAK-4749.patch
>
>
> The jackrabbit-jcr-commons {{ListenerTracker}} collects timing for JCR event
> listeners. It tracks producer (oak internal) and consumer (JCR EventListener)
> time. The initial producer cost is currently not reflected in these stats,
> because {{ChangeProcessor}} in oak-jcr does an initial {{hasNext()}} on the
> {{EventIterator}} outside of the {{ListenerTracker}}. For some listeners this
> initial producer time may even account for the entire cost when the event
> filter rejects all changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4776) Query engine: "like" conditions with only escaped wildcards
[ https://issues.apache.org/jira/browse/OAK-4776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4776. - bulk close for 1.5.10 > Query engine: "like" conditions with only escaped wildcards > --- > > Key: OAK-4776 > URL: https://issues.apache.org/jira/browse/OAK-4776 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.5.10 > > > If a query contains "like" conditions with escaped wildcards, but without > unescaped wildcards, then the result does not include all rows in some cases. > The reason is that such conditions are converted to "=", but the escaped > wildcards are not unescaped. For example, the condition "\[prop\] like > '10\%'" is converted to "\[prop\] = '10\%'" instead of "\[prop\] = '10%'". If > there is an index for that property, the expected result is not found. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4771) Clarify exceptions in DocumentStore
[ https://issues.apache.org/jira/browse/OAK-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4771. - bulk close for 1.5.10 > Clarify exceptions in DocumentStore > --- > > Key: OAK-4771 > URL: https://issues.apache.org/jira/browse/OAK-4771 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4, > resilience > Fix For: 1.6, 1.5.10 > > > The current DocumentStore contract is rather vague about exceptions. The > class JavaDoc mentions implementation specific runtime exceptions, but does > not talk about the DocumentStoreException used by all the DocumentStore > implementations. We should make this explicit in all relevant methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4748) Migration tool should check if the source and destination repositories are different
[
https://issues.apache.org/jira/browse/OAK-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4748.
-
bulk close for 1.5.10
> Migration tool should check if the source and destination repositories are
> different
>
>
> Key: OAK-4748
> URL: https://issues.apache.org/jira/browse/OAK-4748
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Priority: Trivial
> Fix For: 1.6, 1.5.10
>
>
> In case that user passes the same repository as a source and destination, the
> oak-upgrade should fail with appropriate warning. Consider the {{cp}} command
> behaviour:
> {noformat}
> [~]$ cp tmp tmp
> cp: tmp and tmp are identical (not copied).
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4800) Log info message when background operation is successful again
[ https://issues.apache.org/jira/browse/OAK-4800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4800. - bulk close for 1.5.10 > Log info message when background operation is successful again > -- > > Key: OAK-4800 > URL: https://issues.apache.org/jira/browse/OAK-4800 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.6, 1.5.10 > > > The NodeStoreTask responsible for running background operations currently > logs a warn message when the operation fails with an exception. It should log > an info message once the operation is successful again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4758) Allow tweaking of node migration logs threshold
[ https://issues.apache.org/jira/browse/OAK-4758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4758. - bulk close for 1.5.10 > Allow tweaking of node migration logs threshold > --- > > Key: OAK-4758 > URL: https://issues.apache.org/jira/browse/OAK-4758 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: upgrade >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu >Priority: Trivial > Fix For: 1.6, 1.5.10 > > Attachments: oak-upgrade-log.patch > > > Currently upgrade logs each 10k nodes processed. This is hardcoded and I'd > like to provide a simple way of tweaking this value. (ie. set it smaller for > tests, or larger for very large repos that don't really benefit from having a > log each 10k entries). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4723) Optimize PathRev as/from String
[ https://issues.apache.org/jira/browse/OAK-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4723. - bulk close for 1.5.10 > Optimize PathRev as/from String > --- > > Key: OAK-4723 > URL: https://issues.apache.org/jira/browse/OAK-4723 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Labels: performance > Fix For: 1.6, 1.5.10 > > > PathRev instances are used as keys for various cache entries and asString() / > fromString() methods are called frequently when the persistent cache is > enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4639) Enable --missingblobstore by default for the sidegrade
[
https://issues.apache.org/jira/browse/OAK-4639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4639.
-
bulk close for 1.5.10
> Enable --missingblobstore by default for the sidegrade
> --
>
> Key: OAK-4639
> URL: https://issues.apache.org/jira/browse/OAK-4639
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.10
>
>
> The new {{\-\-missingblobstore}} option should be enabled by default for all
> supported cases. It should be only disabled if user specifies
> {{\-\-copy-binaries}} (meaning they're not interested in copying blobs by
> references) and/or the blob store manually (eg. using {{\-\-src-datastore}}).
> Desired outcome should be as follows:
> For sidegrade:
> || src blobstore defined || src blobs embedded || dst blobstore defined ||
> \-\-copy-binaries || outcome src blobstore || outcome action
> | \-| \- | \-| \-
>| missing | copy references^1^
> | \-| \- | \-|
> (/) | missing | (x) not supported
> | \-| \- | (/) | \*
>| missing | (x) not supported
> | \-| (/)| \-| \*
>| embedded | copy to embedded
> | \-| (/)| (/) | \*
>| embedded | copy to defined blobstore
> | (/) | \* | \-| \-
>| as in src | copy references
> | (/) | \* | \-|
> (/) | as in src | copy to embedded
> | (/) | \* | (/) | \*
>| as in src | copy to defined blobstore
> ^1^ - (x) not supported for SegmentMK -> MongoMK migration
> For upgrade:
> || dst blobstore defined || \-\-copy-binaries || outcome src blobstore ||
> outcome action
> | \-| \-| defined by JCR2 |
> copy references
> | \-| (/) | defined by JCR2 |
> copy to embedded
> | (/) | \*| defined by JCR2 |
> copy to defined blobstore
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4769) Update Jackrabbit version to 2.13.3
[ https://issues.apache.org/jira/browse/OAK-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4769. - bulk close for 1.5.10 > Update Jackrabbit version to 2.13.3 > --- > > Key: OAK-4769 > URL: https://issues.apache.org/jira/browse/OAK-4769 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: parent >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.6, 1.5.10 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4725) Enable mmap mode by default for migration to SegmentNodeStore
[
https://issues.apache.org/jira/browse/OAK-4725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4725.
-
bulk close for 1.5.10
> Enable mmap mode by default for migration to SegmentNodeStore
> -
>
> Key: OAK-4725
> URL: https://issues.apache.org/jira/browse/OAK-4725
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: upgrade
>Reporter: Chetan Mehrotra
>Assignee: Tomek Rękawek
>Priority: Minor
> Fix For: 1.6, 1.5.10
>
>
> Currently in {{MigrationOptions}} in upgrade code the mmap option is only
> enabled if explicitly enabled.
> Given it provides much better migration speed it would be better to enable it
> by default (at least for non windows platforms) and provide a way to disable
> it by option. This would ensure that migration uses most suitable options per
> default
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4721) Switch to oak-segment-tar by default when using CLI tools
[
https://issues.apache.org/jira/browse/OAK-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4721.
-
bulk close for 1.5.10
> Switch to oak-segment-tar by default when using CLI tools
> -
>
> Key: OAK-4721
> URL: https://issues.apache.org/jira/browse/OAK-4721
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: run
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.6, 1.5.10
>
>
> CLI tools currently use oak-segment unless the {{\-\-segment-tar}} is
> specified. While this was useful for testing purposes, it should not be
> needed anymore. oak-segment-tar should be used by default,
> {{\-\-segment-tar}} should be removed and a {{\-\-segment}} flag should be
> added to target the old oak-segment implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4755) Change default log of test to info
[ https://issues.apache.org/jira/browse/OAK-4755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4755. - bulk close for 1.5.10 > Change default log of test to info > -- > > Key: OAK-4755 > URL: https://issues.apache.org/jira/browse/OAK-4755 > Project: Jackrabbit Oak > Issue Type: Task > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.6, 1.5.10 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4768) Provide an option to enable Metrics collection for benchmark run
[ https://issues.apache.org/jira/browse/OAK-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4768. - bulk close for 1.5.10 > Provide an option to enable Metrics collection for benchmark run > > > Key: OAK-4768 > URL: https://issues.apache.org/jira/browse/OAK-4768 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.6, 1.5.10 > > > For determining some key metrics I would like to provide an option to > configure metrics for any benchmark run. The StatisticsProvider would be > configured with the respective NodeStore implementation and report would be > added to console -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4722) Adapt the primary and standby commands in oak-run
[
https://issues.apache.org/jira/browse/OAK-4722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4722.
-
bulk close for 1.5.10
> Adapt the primary and standby commands in oak-run
> -
>
> Key: OAK-4722
> URL: https://issues.apache.org/jira/browse/OAK-4722
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: run
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.6, 1.5.10
>
>
> The "primary" and "standby" CLI commands in oak-run are implemented agains
> the oak-tarmk-standby module only. It should be possible to use this command
> both on the new and the old standby implementation, switching between them
> with the {{\-\-segment}} flag. As done for other CLI commands, the backend
> for the command should live in oak-segment-tar.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (OAK-4779) ClusterNodeInfo may renew lease while recovery is running
[ https://issues.apache.org/jira/browse/OAK-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4779. - bulk close for 1.5.10 > ClusterNodeInfo may renew lease while recovery is running > - > > Key: OAK-4779 > URL: https://issues.apache.org/jira/browse/OAK-4779 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: candidate_oak_1_4, resilience > Fix For: 1.6, 1.5.10 > > Attachments: OAK-4779-2.patch, OAK-4779.patch > > > ClusterNodeInfo.renewLease() does not detect when it is being recovered by > another cluster node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4724) Prefetch external changes
[ https://issues.apache.org/jira/browse/OAK-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4724. - bulk close for 1.5.10 > Prefetch external changes > - > > Key: OAK-4724 > URL: https://issues.apache.org/jira/browse/OAK-4724 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, documentmk >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger > Labels: observation > Fix For: 1.6, 1.5.10 > > Attachments: OAK-4724.patch, queue.png > > > In a cluster with listeners that are registered to receive external changes, > pulling in external changes can become a bottleneck. While processing those > external changes, further local changes are put into the observation queue > leading to a system where the queue eventually fills up. > Instead of processing external changes one after another, the implementation > could prefetch them as they come in and if needed pull them in parallel. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4764) Update httpclient to 4.3.6
[ https://issues.apache.org/jira/browse/OAK-4764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4764. - bulk close for 1.5.10 > Update httpclient to 4.3.6 > -- > > Key: OAK-4764 > URL: https://issues.apache.org/jira/browse/OAK-4764 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: solr >Reporter: Tommaso Teofili >Assignee: Tommaso Teofili > Fix For: 1.5.10 > > > It'd be good to update to latest version of httpclient (4.3.6) to incorporate > the latest fixes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4794) RDBDocumentStore: update PostgresQL JDBC driver
[ https://issues.apache.org/jira/browse/OAK-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella closed OAK-4794. - bulk close for 1.5.10 > RDBDocumentStore: update PostgresQL JDBC driver > --- > > Key: OAK-4794 > URL: https://issues.apache.org/jira/browse/OAK-4794 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.33, 1.2.18, 1.4.7, 1.5.9 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.0.34, 1.5.10, 1.4.8, 1.2.20 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (OAK-4770) Missing exception handling in ClusterNodeInfo.renewLease()
[
https://issues.apache.org/jira/browse/OAK-4770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella closed OAK-4770.
-
bulk close for 1.5.10
> Missing exception handling in ClusterNodeInfo.renewLease()
> --
>
> Key: OAK-4770
> URL: https://issues.apache.org/jira/browse/OAK-4770
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, documentmk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
> Labels: candidate_oak_1_4, resilience
> Fix For: 1.6, 1.5.10
>
>
> ClusterNodeInfo.renewLease() does not handle a potential
> DocumentStoreException on {{findAndModify()}}. This may leave
> {{previousLeaseEndTime}} in an inconsistent state and a subsequent
> {{renewLease()}} call then considers the lease timed out.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4043:
-
Labels: (was: candidate_oak_1_4)
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Fix For: 1.6, 1.4.8, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes (OAK-2749), which we already did in AEM, but the checkpoint
> tool is blissfully unaware of this and it might trigger a full reindex
> following offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4043:
-
Fix Version/s: 1.4.8
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Fix For: 1.6, 1.4.8, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes (OAK-2749), which we already did in AEM, but the checkpoint
> tool is blissfully unaware of this and it might trigger a full reindex
> following offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502911#comment-15502911
]
Alex Parvulescu edited comment on OAK-4043 at 9/19/16 10:09 AM:
fixed with http://svn.apache.org/viewvc?rev=1761412=rev
merged to 1.4 with http://svn.apache.org/viewvc?rev=1761414=rev
was (Author: alex.parvulescu):
fixed with http://svn.apache.org/viewvc?rev=1761412=rev
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Fix For: 1.6, 1.4.8, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes (OAK-2749), which we already did in AEM, but the checkpoint
> tool is blissfully unaware of this and it might trigger a full reindex
> following offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4731) Refine forced compaction
[
https://issues.apache.org/jira/browse/OAK-4731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486644#comment-15486644
]
Michael Dürig edited comment on OAK-4731 at 9/19/16 9:58 AM:
-
The option to force compaction in {{SegmentNodeStoreService}} has been replaced
by an option to set a time out on forced compaction
({{compaction.force.timeout}}). That time out specifies the number of seconds
to attempt forced compaction. This includes the time it takes to acquire a
exclusive write lock on the node store and the compaction time itself. Forced
compaction will not be attempted if the time out value is set to 0. The
default value is 60 seconds.
was (Author: mduerig):
The option to force compaction in {{SegmentNodeStoreService}} has been replaced
by an option to set a time out on forced compaction
({{compaction.force.timeout}}). That time out specifies the number of seconds
to attempt forced compaction. This includes the time it takes to acquire a
exclusive write lock on the node store and the compaction time itself. Forced
compaction will not be attempted if the time out value is set to 0.
> Refine forced compaction
>
>
> Key: OAK-4731
> URL: https://issues.apache.org/jira/browse/OAK-4731
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: compaction, gc
> Fix For: Segment Tar 0.0.12
>
>
> Forced compaction currently acquires an exclusive write lock on the
> repository blocking all concurrent commits during the complete time it needs
> to finish compaction. I think we should refine this:
> * Add a time out so we could limit the time during which the repository does
> not accept writes while still giving compaction another chance to finish.
> * Boost the compaction threads priority. This could actually already be done
> during the regular compaction cycles to increase the changes to finish in
> time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502913#comment-15502913
]
Alex Parvulescu commented on OAK-4043:
--
bq. Alex Parvulescu In addition to this we should add a check in
Oak#withAsyncIndexing(java.lang.String, long) to enforce this convention around
checkpoint name i.e. name ending with async
I think that's a different concern and needs to be tracked in a dedicated issue
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Labels: candidate_oak_1_4
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes (OAK-2749), which we already did in AEM, but the checkpoint
> tool is blissfully unaware of this and it might trigger a full reindex
> following offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4043:
-
Component/s: (was: core)
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Labels: candidate_oak_1_4
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes (OAK-2749), which we already did in AEM, but the checkpoint
> tool is blissfully unaware of this and it might trigger a full reindex
> following offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu resolved OAK-4043.
--
Resolution: Fixed
fixed with http://svn.apache.org/viewvc?rev=1761412=rev
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Labels: candidate_oak_1_4
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes (OAK-2749), which we already did in AEM, but the checkpoint
> tool is blissfully unaware of this and it might trigger a full reindex
> following offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502908#comment-15502908
]
Alex Parvulescu commented on OAK-4043:
--
bq. The method in the CheckpointsHelper could be simplified using
Utils.min(Revision, Revision)
thanks for the suggestion! did not know about this method :)
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Labels: candidate_oak_1_4
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes (OAK-2749), which we already did in AEM, but the checkpoint
> tool is blissfully unaware of this and it might trigger a full reindex
> following offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4818) Version and Version History nodes don't implement the proper interfaces when found via Node.getNodes()
[
https://issues.apache.org/jira/browse/OAK-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502873#comment-15502873
]
Michael Dürig commented on OAK-4818:
Most of those exceptions should never actually occur on that code path. The
others are {{IllegalItemStateException}} instances indicating the iterator is
not valid any more. This suggest that we could as well throw an
{{IllegaleStateException}} instead of logging and carrying on (like we already
do e.g. in {{NodeImpl#getNodes()}}).
But I agree, there is not silver bullet here. This is one of the places where
the JCR API puts us into a hard place.
> Version and Version History nodes don't implement the proper interfaces when
> found via Node.getNodes()
> --
>
> Key: OAK-4818
> URL: https://issues.apache.org/jira/browse/OAK-4818
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: jcr
>Affects Versions: 1.6, 1.0.33, 1.2.19, 1.5.10
>Reporter: Csaba Varga
>Priority: Minor
> Attachments: version-traversal-issue.patch
>
>
> The JCR spec says in section 15.6:
> When an nt:versionHistory or nt:version node is acquired through a query or
> directly through a getNode, the actual Java type of the returned object must
> be VersionHistory (in the case nt:versionHistory nodes) or Version (in the
> case of nt:version nodes).
> While this doesn't explicitly mention the getNodes() method, I believe it's a
> reasonable expectation that this method should work the same way, i.e. make
> the actual Java type of the returned child object Version or VersionHistory
> where applicable. Oak currently doesn't work this way; the iterator returned
> by either getNodes() overload will always yield NodeImpl instances.
> I will attach a suggested patch to fix this, including addition to the unit
> tests to catch any regressions in the future. The patch is against the latest
> trunk, but the same behavior seems to be present in all past versions as
> well, so if it's not a big problem, could you also backport this to older
> stable versions as well? (My employer is using AEM 6.1 currently, so we would
> be interested in getting this backported to 1.2 at least.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4759) Queueless change processor
[
https://issues.apache.org/jira/browse/OAK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger updated OAK-4759:
--
Attachment: OAK-4759.patch
jackrabbit-api.patch
Changed the proposal to use a marker interface. Using a flag on the filter
previously allowed the listener to reregister and switch between a processor
with a queue and one without. That would have been more tricky to implement.
> Queueless change processor
> --
>
> Key: OAK-4759
> URL: https://issues.apache.org/jira/browse/OAK-4759
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, documentmk, jcr
>Reporter: Marcel Reutegger
> Labels: observation
> Fix For: 1.6
>
> Attachments: OAK-4759.patch, OAK-4759.patch, jackrabbit-api.patch,
> jackrabbit-api.patch
>
>
> The initial proposal for this improvement was:
> {quote}
> Change processing for listeners that are only interested in external
> events could be simplified because there is no commit info for
> external changes. The basic idea is that the node store implementation
> may be able to optimize batch processing of multiple external changes
> and does not need to process each external change individually. The
> DocumentNodeStore would use the journal to identify external changes
> and need to come up with a way to ignore overlapping local changes.
> With this new feature, expensive listeners that process local as well
> as external events could be split into two separate listeners, each
> optimized for the type of changes.
> {quote}
> Later the proposal was changed in a more general queueless change processor.
> See first comment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4043:
-
Description:
Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a single
checkpoint reference (the default one). Now is it possible to add multiple
lanes (OAK-2749), which we already did in AEM, but the checkpoint tool is
blissfully unaware of this and it might trigger a full reindex following
offline compaction.
fyi [~edivad], [~chetanm]
[0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
was:
Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a single
checkpoint reference (the default one). Now is it possible to add multiple
lanes, which we already did in AEM, but the checkpoint tool is blissfully
unaware of this and it might trigger a full reindex following offline
compaction.
fyi [~edivad], [~chetanm]
[0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Labels: candidate_oak_1_4
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes (OAK-2749), which we already did in AEM, but the checkpoint
> tool is blissfully unaware of this and it might trigger a full reindex
> following offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4818) Version and Version History nodes don't implement the proper interfaces when found via Node.getNodes()
[ https://issues.apache.org/jira/browse/OAK-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502834#comment-15502834 ] Marcel Reutegger commented on OAK-4818: --- Thanks for reporting this issue. In general, the patch looks good to me. Handling checked exceptions in functions is indeed a problem and falling back to NodeImpl might be the best we can do here. [~mduerig], WDYT? > Version and Version History nodes don't implement the proper interfaces when > found via Node.getNodes() > -- > > Key: OAK-4818 > URL: https://issues.apache.org/jira/browse/OAK-4818 > Project: Jackrabbit Oak > Issue Type: Bug > Components: jcr >Affects Versions: 1.6, 1.0.33, 1.2.19, 1.5.10 >Reporter: Csaba Varga >Priority: Minor > Attachments: version-traversal-issue.patch > > > The JCR spec says in section 15.6: > When an nt:versionHistory or nt:version node is acquired through a query or > directly through a getNode, the actual Java type of the returned object must > be VersionHistory (in the case nt:versionHistory nodes) or Version (in the > case of nt:version nodes). > While this doesn't explicitly mention the getNodes() method, I believe it's a > reasonable expectation that this method should work the same way, i.e. make > the actual Java type of the returned child object Version or VersionHistory > where applicable. Oak currently doesn't work this way; the iterator returned > by either getNodes() overload will always yield NodeImpl instances. > I will attach a suggested patch to fix this, including addition to the unit > tests to catch any regressions in the future. The patch is against the latest > trunk, but the same behavior seems to be present in all past versions as > well, so if it's not a big problem, could you also backport this to older > stable versions as well? (My employer is using AEM 6.1 currently, so we would > be interested in getting this backported to 1.2 at least.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502757#comment-15502757
]
Marcel Reutegger commented on OAK-4043:
---
Patch looks good to me. The method in the CheckpointsHelper could be simplified
using Utils.min(Revision, Revision), but the method was only introduced
recently (1.4). That is, the current patch is easier to backport to 1.2 and
earlier if necessary.
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Labels: candidate_oak_1_4
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes, which we already did in AEM, but the checkpoint tool is
> blissfully unaware of this and it might trigger a full reindex following
> offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4043) Oak run checkpoints needs to account for multiple index lanes
[
https://issues.apache.org/jira/browse/OAK-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502478#comment-15502478
]
Chetan Mehrotra commented on OAK-4043:
--
Patch look fine. Without this patch {{rm-unreferenced}}
{noformat}
$java -jar oak-run-1.6-SNAPSHOT.jar checkpoints --segment=true
/path/to/segmentstore rm-unreferenced
Apache Jackrabbit Oak 1.6-SNAPSHOT
Checkpoints /path/to/segmentstore
Referenced checkpoint from /:async@async is bcda394b-9a01-45e2-a326-bab1618ac60e
Removed 1 checkpoints in 112ms.
{noformat}
With this fix
{noformat}
$ java -jar oak-run-1.6-SNAPSHOT.jar checkpoints --segment=true
/path/to/segmentstore rm-unreferenced
Apache Jackrabbit Oak 1.6-SNAPSHOT
Checkpoints /path/to/segmentstore
Referenced checkpoint from /:async@async is bcda394b-9a01-45e2-a326-bab1618ac60e
Referenced checkpoint from /:async@fulltext-async is
46dce134-fe32-4929-84bf-abbd0ed46d9d
Removed 0 checkpoints in 65ms.
{noformat}
[~mreutegg] Just to be double safe here can you also review the changes around
checkpoint handling for DocumentNodeStore
[~alex.parvulescu] In addition to this we should add a check in
{{Oak#withAsyncIndexing(java.lang.String, long)}} to enforce this convention
around checkpoint name i.e. name ending with {{async}}
> Oak run checkpoints needs to account for multiple index lanes
> -
>
> Key: OAK-4043
> URL: https://issues.apache.org/jira/browse/OAK-4043
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, run
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Critical
> Labels: candidate_oak_1_4
> Fix For: 1.6, 1.5.11
>
> Attachments: OAK-4043.patch
>
>
> Oak run {{checkpoints rm-unreferenced}} [0] currently is hardcoded on a
> single checkpoint reference (the default one). Now is it possible to add
> multiple lanes, which we already did in AEM, but the checkpoint tool is
> blissfully unaware of this and it might trigger a full reindex following
> offline compaction.
> fyi [~edivad], [~chetanm]
> [0] https://github.com/apache/jackrabbit-oak/tree/trunk/oak-run#checkpoints
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
