[jira] [Commented] (OAK-5944) VersionGarbageCollector: allow to update _deletedOnce in batch operations

2017-03-20 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15932372#comment-15932372 ] Stefan Eissing commented on OAK-5944: - My measurements (dev laptop, SSD, local 3.4 mongo), 100GB aem

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-03-17 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929996#comment-15929996 ] Stefan Eissing commented on OAK-4780: - And I will make two patches, one for oak-core and one for oak-run

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-03-17 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929993#comment-15929993 ] Stefan Eissing commented on OAK-4780: - bq. VersionGarbageCollector.reset() can be simplified with just

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-03-10 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904841#comment-15904841 ] Stefan Eissing commented on OAK-4780: - Result from run of current version against a large customer

[jira] [Commented] (OAK-3070) Use a lower bound in VersionGC query to avoid checking unmodified once deleted docs

2017-03-09 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902866#comment-15902866 ] Stefan Eissing commented on OAK-3070: - Can a [partial

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-03-08 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901533#comment-15901533 ] Stefan Eissing commented on OAK-4780: - Updated

[jira] [Commented] (OAK-5368) Not configurable/Unnecessary short Lucene Observation Queue length

2017-03-08 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901046#comment-15901046 ] Stefan Eissing commented on OAK-5368: - You are correct. I must have scanned for creations of

[jira] [Commented] (OAK-5368) Not configurable/Unnecessary short Lucene Observation Queue length

2017-03-08 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901000#comment-15901000 ] Stefan Eissing commented on OAK-5368: - [~chetanm] unfortunately I do not have the logs any longer.

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-03-07 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899710#comment-15899710 ] Stefan Eissing commented on OAK-4780: - * I will merge current trunk when you have added your OAK-3070

[jira] [Comment Edited] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-03-03 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894387#comment-15894387 ] Stefan Eissing edited comment on OAK-4780 at 3/3/17 1:51 PM: - Updated my github

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-03-03 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894387#comment-15894387 ] Stefan Eissing commented on OAK-4780: - Updated my github clone with the following: * configure

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-03-01 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890290#comment-15890290 ] Stefan Eissing commented on OAK-4780: - Here

[jira] [Commented] (OAK-3070) Use a lower bound in VersionGC query to avoid checking unmodified once deleted docs

2017-02-24 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882605#comment-15882605 ] Stefan Eissing commented on OAK-3070: - Apart from OAK-3070, resetting {{_deletedOnce}} keeps indices

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-02-21 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15875897#comment-15875897 ] Stefan Eissing commented on OAK-4780: - [~reschke] any estimation how long that "eventually" will take on

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-02-20 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874699#comment-15874699 ] Stefan Eissing commented on OAK-4780: - Assuming there is the following in place: 1. Time/Revision of

[jira] [Updated] (OAK-5571) VersionGarbageCollector can remove leaf nodes eagerly

2017-02-17 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Eissing updated OAK-5571: Attachment: clean_leaves.patch Patch that clears the leaves collection after batch remove. >

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-02-17 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871558#comment-15871558 ] Stefan Eissing commented on OAK-4780: - [~reschke] With OAK-5571 committed in trunk, this one can be

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-02-02 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849819#comment-15849819 ] Stefan Eissing commented on OAK-4780: - +1 for OAK-5571 In regards to resilience, the VGC needs

[jira] [Updated] (OAK-5433) System Pacing Service

2017-01-26 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Eissing updated OAK-5433: Attachment: obs-pacing-v3.diff ing-6k-v4-500-7500-wfq.png Enhanced version that goes

[jira] [Commented] (OAK-5433) System Pacing Service

2017-01-25 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837745#comment-15837745 ] Stefan Eissing commented on OAK-5433: - [~egli] Thanks, in that case, the proposed implementation is

[jira] [Commented] (OAK-5433) System Pacing Service

2017-01-25 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837612#comment-15837612 ] Stefan Eissing commented on OAK-5433: - Hmm, not sure of your interpretation of the {{PacingService}}. My

[jira] [Commented] (OAK-5446) Lease Impossible to Renew once Failure Margin Passed

2017-01-25 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837458#comment-15837458 ] Stefan Eissing commented on OAK-5446: - bq. I second Julian Reschke's finding that ClusterNodeInfo gets

[jira] [Updated] (OAK-5433) System Pacing Service

2017-01-20 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Eissing updated OAK-5433: Attachment: ing-6k-t500-7500-6-wflauncher.png Sample run with the configured

[jira] [Updated] (OAK-5433) System Pacing Service

2017-01-20 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Eissing updated OAK-5433: Attachment: oak-trunk-pacer-v1.diff A proposal for an introduction of a {{Pacer}} that can be

[jira] [Comment Edited] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-01-19 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829882#comment-15829882 ] Stefan Eissing edited comment on OAK-4780 at 1/19/17 1:07 PM: -- Changes in

[jira] [Updated] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-01-19 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Eissing updated OAK-4780: Attachment: leafnodes-v3.diff Changes: # removed System.err.println() # using google collection

[jira] [Created] (OAK-5488) BackgroundObserver MBean report Listener class again

2017-01-19 Thread Stefan Eissing (JIRA)
Stefan Eissing created OAK-5488: --- Summary: BackgroundObserver MBean report Listener class again Key: OAK-5488 URL: https://issues.apache.org/jira/browse/OAK-5488 Project: Jackrabbit Oak Issue

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-01-18 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828405#comment-15828405 ] Stefan Eissing commented on OAK-4780: - [~mreutegg], thanks for the review. I clearly lay the blame of

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-01-18 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828227#comment-15828227 ] Stefan Eissing commented on OAK-4780: - Ah, yes. worked in the change that split leaf nodes do not get

[jira] [Updated] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-01-18 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Eissing updated OAK-4780: Attachment: leafnodes-v2.diff My take on this. Patch against oak/trunk, integrationTest run with a

[jira] [Commented] (OAK-4780) VersionGarbageCollector should be able to run incrementally

2017-01-18 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827716#comment-15827716 ] Stefan Eissing commented on OAK-4780: - Since Julian is currently busy with other things, I will give it

[jira] [Commented] (OAK-5433) System Pacing Service

2017-01-11 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15818168#comment-15818168 ] Stefan Eissing commented on OAK-5433: - Agreed. So, if we make the new service separate and optional, we

[jira] [Commented] (OAK-5433) System Pacing Service

2017-01-11 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15818093#comment-15818093 ] Stefan Eissing commented on OAK-5433: - I do not think there would be a performance issue and those could

[jira] [Commented] (OAK-5433) System Pacing Service

2017-01-10 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815192#comment-15815192 ] Stefan Eissing commented on OAK-5433: - The tests were done on a patched 1.4.9. The cost when the queue

[jira] [Commented] (OAK-5433) System Pacing Service

2017-01-10 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814957#comment-15814957 ] Stefan Eissing commented on OAK-5433: - Some more explanation to the patch: # it also experimented around

[jira] [Updated] (OAK-5433) System Pacing Service

2017-01-10 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Eissing updated OAK-5433: Attachment: obs-pacing.diff The patch used in the tests. This is a PoC rough implementation, of

[jira] [Created] (OAK-5433) System Pacing Service

2017-01-10 Thread Stefan Eissing (JIRA)
Stefan Eissing created OAK-5433: --- Summary: System Pacing Service Key: OAK-5433 URL: https://issues.apache.org/jira/browse/OAK-5433 Project: Jackrabbit Oak Issue Type: New Feature

[jira] [Commented] (OAK-5368) Not configurable/Unnecessary short Lucene Observation Queue length

2016-12-23 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15772404#comment-15772404 ] Stefan Eissing commented on OAK-5368: - [~chetanm] I agree that it will work for the compacted changes

[jira] [Updated] (OAK-5368) Not configurable/Unnecessary short Lucene Observation Queue length

2016-12-22 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Eissing updated OAK-5368: Attachment: LuceneIndexConfigObserver-under-load.png Sample measurement of a patched

[jira] [Created] (OAK-5368) Not configurable/Unnecessary short Lucene Observation Queue length

2016-12-22 Thread Stefan Eissing (JIRA)
Stefan Eissing created OAK-5368: --- Summary: Not configurable/Unnecessary short Lucene Observation Queue length Key: OAK-5368 URL: https://issues.apache.org/jira/browse/OAK-5368 Project: Jackrabbit Oak

[jira] [Commented] (OAK-5238) IndexCopier causes concurrent update on NodeBuilder

2016-12-08 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15732180#comment-15732180 ] Stefan Eissing commented on OAK-5238: - And if one created/initialized and commited a subnode, took a

[jira] [Commented] (OAK-3606) Improvements for IndexStatsMBean usage

2016-10-25 Thread Stefan Eissing (JIRA)
[ https://issues.apache.org/jira/browse/OAK-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15605235#comment-15605235 ] Stefan Eissing commented on OAK-3606: - So, for a particular index, the {{LastIndexedTime}} is only