[jira] [Commented] (OAK-3767) define index in subdirectories of oak:index

[Thomas Mueller (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052557#comment-15052557
 ] 

Thomas Mueller commented on OAK-3767:
-

> Shall we allow this only under root or under any arbitrary path as well? For 
> example /content

Personally, I don't think it is very important for arbitrary path. Even if it 
turns out it's easy to implement, I'm not sure if we should implement it, 
because it complicates the documentation (it increases the complexity from the 
end user side of view).

> define index in subdirectories of oak:index
> ---
>
> Key: OAK-3767
> URL: https://issues.apache.org/jira/browse/OAK-3767
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core, query
>Reporter: Davide Giannella
>
> Provide the ability to define indexes in subdirectories of an oak:index node. 
> Something like
> {noformat}
> /oak:index
>  -> oak
>  -> apps
> {noformat}
> *question*. Shall we allow this only under root or under any arbitrary path 
> as well? For example /content.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3768) Remove OrderedPropertyIndex support from trunk

[Chetan Mehrotra (JIRA)]

Chetan Mehrotra created OAK-3768:


 Summary: Remove OrderedPropertyIndex support from trunk
 Key: OAK-3768
 URL: https://issues.apache.org/jira/browse/OAK-3768
 Project: Jackrabbit Oak
  Issue Type: Task
  Components: query
Reporter: Chetan Mehrotra
Assignee: Davide Giannella
 Fix For: 1.4


As discussed [1] we should remove code related to ordered property index 

[1] http://markmail.org/thread/drswaq3tccv4nthi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3139) SNFE in persisted comapation map when using CLEAN_OLD

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3139:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> SNFE in persisted comapation map when using CLEAN_OLD
> -
>
> Key: OAK-3139
> URL: https://issues.apache.org/jira/browse/OAK-3139
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.4
>
> Attachments: OAK-3139.patch
>
>
> When using {{CLEAN_OLD}} it might happen that segments of the persisted 
> compaction map get collected. --The reason for this is that only the segment 
> containing the root of the map is pinned ({{SegmentId#pin}}), leaving other 
> segments of the compaction map eligible for collection once old enough.--
> {noformat}
> org.apache.jackrabbit.oak.plugins.segment.SegmentNotFoundException: Segment 
> 95cbb3e2-3a8c-4976-ae5b-6322ff102731 not found
> at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.readSegment(FileStore.java:919)
> at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.getSegment(SegmentTracker.java:134)
> at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:108)
> at 
> org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
> at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:154)
> at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:186)
> at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.get(PersistedCompactionMap.java:118)
> at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.get(PersistedCompactionMap.java:100)
> at 
> org.apache.jackrabbit.oak.plugins.segment.CompactionMap.get(CompactionMap.java:93)
> at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.uncompact(SegmentWriter.java:1023)
> at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1033)
> at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:100)
> at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore$Commit.(SegmentNodeStore.java:418)
> at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.merge(SegmentNodeStore.java:204)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3264) Deadlock between persisted compaction map and cleanup 2

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3264:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 cleanup compaction gc  (was: 
cleanup compaction gc)

> Deadlock between persisted compaction map and cleanup 2
> ---
>
> Key: OAK-3264
> URL: https://issues.apache.org/jira/browse/OAK-3264
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, 
> compaction, gc
> Fix For: 1.3.5
>
>
> Just seen this deadlock while running {{SegmentCompactionIT}}:
> {noformat}
> "pool-1-thread-47":
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.readSegment(FileStore.java:910)
>   - waiting to lock <0x000700110bd0> (a 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.readSegment(SegmentTracker.java:211)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:149)
>   - locked <0x000700328b88> (a 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:154)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.get(PersistedCompactionMap.java:121)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.get(PersistedCompactionMap.java:103)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.CompactionMap.get(CompactionMap.java:93)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.uncompact(SegmentWriter.java:1074)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1098)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:100)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:85)
>   at 
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.updated(MemoryNodeBuilder.java:214)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:81)
>   at 
> org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.remove(MemoryNodeBuilder.java:355)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentCompactionIT$RandomWriter.modify(SegmentCompactionIT.java:448)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentCompactionIT$RandomWriter.call(SegmentCompactionIT.java:430)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentCompactionIT$RandomWriter.call(SegmentCompactionIT.java:406)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run(Thread.java:745)
> "TarMK flush thread [target/SegmentCompactionIT9065337410200765612dir], 
> active since Fri Aug 21 06:53:18 GMT+00:00 2015, previous max duration 
> 40846ms":
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:145)
>   - waiting to lock <0x000700328b88> (a 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:154)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.compress(PersistedCompactionMap.java:204)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.remove(PersistedCompactionMap.java:155)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.CompactionMap.remove(CompactionMap.java:108)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.cleanup(FileStore.java:699)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.flush(FileStore.java:628)
>   - locked <0x000700110bd0> (a 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
>   - locked 

[jira] [Updated] (OAK-3347) Ineffective cleanup after compaction due to references to root

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3347:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 cleanup compac gc  (was: 
cleanup compac gc)

> Ineffective cleanup after compaction due to references to root
> --
>
> Key: OAK-3347
> URL: https://issues.apache.org/jira/browse/OAK-3347
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, compac, gc
> Fix For: 1.3.6
>
>
> The cleanup phase after a compaction run is currently not able to remove the 
> pre compacted segments as the previous (pre-compacted) root is still being 
> referenced. Those references are coming from:
> * The {{before}} [local 
> variable|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/file/FileStore.java#L653]
>  in {{FileStore.flush}}.
> * The [current 
> head|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentNodeStore.java#L85-L85]
>  of the {{SegmentNodeStore}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3529) NodeStore API should expose an Instance ID

[Davide Giannella (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-3529:
--
Attachment: OAK-3529-5.patch

Attaching a [new patch|^OAK-3529-5.patch] where I removed the
getInstaceId from the builders.

The Jcr builder now keep tracks of the instanceId and NodeStore.

I personally don't like it and I think we should address the situation
by adding a new {{.with(@Nonnull Clusterable)}}. The Clusterable
aspect could be implemented by something different than a
NodeStore. In particular deployments for example someone could create
a "service" that delegates it to Sling rather than our NodeStore or we
could come up with type of Topology which we could later on leverage.

By specifying the new .with() we make it clear from a repository
construction POV.

This new {{.with()}} can be addressed later on with a separate issue.

[~mduerig], [~mreutegg] please review

> NodeStore API should expose an Instance ID
> --
>
> Key: OAK-3529
> URL: https://issues.apache.org/jira/browse/OAK-3529
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.4
>
> Attachments: OAK-3529-1.patch, OAK-3529-2.patch, OAK-3529-3.patch, 
> OAK-3529-4.patch, OAK-3529-5.patch
>
>
> For better leveraging cluster oriented algorithm: discovery, atomic
> operations; it would be very helpful if the NodeStore could expose a
> unique instance id.
> This can be the same as a cluster ID.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052669#comment-15052669
 ] 

Tomek Rękawek commented on OAK-3743:


Even with the latest changes applied, the jobs still times out on certain 
machines. Interesting fact: tests in the {{oak-core}} are not affected by the 
issue, it's the {{oak-jcr}} package that times out (taking 1:15h rather than 8 
minutes).

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3766) Investigate and remove dependencies from oak-run

[Davide Giannella (JIRA)]

Davide Giannella created OAK-3766:
-

 Summary: Investigate and remove dependencies from oak-run
 Key: OAK-3766
 URL: https://issues.apache.org/jira/browse/OAK-3766
 Project: Jackrabbit Oak
  Issue Type: Task
Reporter: Davide Giannella


While with OAK-3134 the effort is to split oak-run into smaller bundle to ease 
deployment and maintenance this is to see if we can easily cut the size of it 
by removing unused dependencies.

*dependencies identified so far to investigate*

- hadoop
- solr
- tika
- jetty
- zookeeper
- H2 (SQL part)
- JR remoting



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3765) Parallelized test runner does not wait for test completion

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052522#comment-15052522
 ] 

Michael Dürig commented on OAK-3765:


bq. In the attached patch, the finished() method waits indefinitely until all 
tests are done.

+1 to this behaviour as this makes it clearer where an actual problem is.

Maybe there is a way here to dump stack traces to a log for tests that don't 
terminate after a given time!?

> Parallelized test runner does not wait for test completion
> --
>
> Key: OAK-3765
> URL: https://issues.apache.org/jira/browse/OAK-3765
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: jcr
>Affects Versions: 1.2.9, 1.0.25, 1.3.12
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
>
> As analyzed by [~tomek.rekawek] in OAK-3743:
> Quote from the mailing list:
> {quote}
> On a “slow” machine, in the surefire logs for the AtomicCounterTest (which 
> takes 63 sec while it should 3 sec), following test case appears:
> {noformat}
>classname="org.apache.jackrabbit.oak.jcr.RepositoryTest" 
> name="importWithRegisteredType[RDBDocumentStore on 
> jdbc:derby:oaktest\;create=true]"/>
> {noformat}
> It’s a test case from a completely different class. I downloaded all the 
> surefire reports and it seems that the test cases from the RepositoryTest 
> class are spread across CRUDTest, ConflictResolutionTest, ObservationTest and 
> others. On the “fast” machines the problem doesn’t exist and all 
> RepositoryTest methods are invoked within their own test case.
> {quote}
> It seems that this behaviour is caused by the 
> {{org.apache.jackrabbit.oak.jcr.Parallelized}} runner, used in the 
> {{AbstractRepositoryTest}}. JUnit submits a {{Runnable}} wrapping all the 
> test cases from a single class to the {{ThreadPoolScheduler#schedule()}} 
> method and then invokes {{#finished()}}. The latter waits for 10 minutes 
> until the submitted {{Runnable}} is done. However, in case of the 
> {{RepositoryTest}}, all tests takes more than 10 minutes. The {{finished()}} 
> method simply returns, JUnit invokes another test class, while the 
> {{RepositoryTest}} is still running in the background.
> In the attached patch, the {{finished()}} method waits indefinitely until all 
> tests are done. It may seem a bit radical, but after all it's the same 
> behaviour as in single-thread JUnit runner, which doesn't have internal 
> timeout as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3705) Change default of compaction.forceAfterFail to false

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3705:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Change default of compaction.forceAfterFail to false
> 
>
> Key: OAK-3705
> URL: https://issues.apache.org/jira/browse/OAK-3705
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.12
>
>
> Having {{compaction.forceAfterFail}} set to {{true}} will block repository 
> writes for an extended period of time (minutes, probably hours) if all 
> previous compaction cycles couldn't catch up with the latest changes. I think 
> this is not acceptable and we should change the default to {{false}}: if 
> compaction is not able to catch up the recommendation should be to move it to 
> a quieter time. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3094) Potential ClassCastException with LIRS cache builder

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3094:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> Potential ClassCastException with LIRS cache builder
> 
>
> Key: OAK-3094
> URL: https://issues.apache.org/jira/browse/OAK-3094
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.3
>
>
> {{CacheLIRS.Builder}} does unsafe casts, which can lead to {{CCE}} later on. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-2862) CompactionMap#compress() inefficient for large compaction maps

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-2862:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction doc-impacting gc  
(was: compaction doc-impacting gc)

> CompactionMap#compress() inefficient for large compaction maps
> --
>
> Key: OAK-2862
> URL: https://issues.apache.org/jira/browse/OAK-2862
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, 
> doc-impacting, gc
> Fix For: 1.3.0
>
> Attachments: OAK-2862-memory.png, OAK-2862.png, benchLargeMap.xlsx
>
>
> I've seen {{CompactionMap#compress()}} take up most of the time spent in 
> compaction. With 40M record ids in the compaction map compressing runs for 
> hours. 
> I will back this with numbers as soon as I have a better grip on the issue.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-2734) Compaction does not finish on repository with continuous writes

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-2734:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction doc-impacting gc  
(was: compaction doc-impacting gc)

> Compaction does not finish on repository with continuous writes 
> 
>
> Key: OAK-2734
> URL: https://issues.apache.org/jira/browse/OAK-2734
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, 
> doc-impacting, gc
> Fix For: 1.3.0
>
>
> A repository with continuous writes can keep the compactor from completing 
> causing the repository size to grow indefinitely. 
> This effect is caused by the compactor trying to catch up with changes that 
> occurred while compacting. I.e. compacting them on top of the already 
> compacted head. When there is a steady stream of incoming changes it can 
> happen that the compactor never actually catches up. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3007) SegmentStore cache does not take "string" map into account

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3007:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 doc-impacting resilience 
scalability  (was: doc-impacting resilience scalability)

> SegmentStore cache does not take "string" map into account
> --
>
> Key: OAK-3007
> URL: https://issues.apache.org/jira/browse/OAK-3007
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Thomas Mueller
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, doc-impacting, 
> resilience, scalability
> Fix For: 1.3.3
>
> Attachments: OAK-3007-2.patch, OAK-3007-3.patch, OAK-3007.patch
>
>
> The SegmentStore cache size calculation ignores the size of the field 
> Segment.string (a concurrent hash map). It looks like a regular segment in a 
> memory mapped file has the size 1024, no matter how many strings are loaded 
> in memory. This can lead to out of memory. There seems to be no way to limit 
> (configure) the amount of memory used by strings. In one example, 100'000 
> segments are loaded in memory, and 5 GB are used for Strings in that map.
> We need a way to configure the amount of memory used for that. This seems to 
> be basically a cache. OAK-2688 does this, but it would be better to have one 
> cache with a configurable size limit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-2849) Improve revision gc on SegmentMK

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-2849:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Improve revision gc on SegmentMK
> 
>
> Key: OAK-2849
> URL: https://issues.apache.org/jira/browse/OAK-2849
> Project: Jackrabbit Oak
>  Issue Type: Epic
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.4
>
> Attachments: SegmentCompactionIT-conflicts.png
>
>
> This is a container issue for the ongoing effort to improve revision gc of 
> the SegmentMK. 
> I'm exploring 
> * ways to make the reference graph as exact as possible and necessary: it 
> should not contain segments that are not referenceable any more and but must 
> contain all segments that are referenceable. 
> * ways to segregate the reference graph reducing dependencies between certain 
> set of segments as much as possible. 
> * Reducing the number of in memory references and their impact on gc as much 
> as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3134) Identify functionality offered by oak-run

[Davide Giannella (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052508#comment-15052508
 ] 

Davide Giannella commented on OAK-3134:
---

I think we can definitely get rid of most of the dependencies you highlighted. 
Going to file a separate issue for that.

> Identify functionality offered by oak-run
> -
>
> Key: OAK-3134
> URL: https://issues.apache.org/jira/browse/OAK-3134
> Project: Jackrabbit Oak
>  Issue Type: Epic
>  Components: run
>Reporter: Davide Giannella
>Assignee: Davide Giannella
>
> oak-run reached the size of 50MB+ and offers indeed various functionalities 
> that could be moved to their own module.
> This ticket is about to identify what oak-run currently offers and what/if 
> could be split.
> ML thread: http://markmail.org/thread/w34bphrk57l7pkaz
> || Functionality || Package || Module ||
> | micro-benchmark | org.apache.jackrabbit.oak.benchmark |oak-development |
> | scalability benchmark | org.apache.jackrabbit.oak.scalability | 
> oak-development|
> | Groovy console | org.apache.jackrabbit.oak.console, 
> /oak-run/src/main/groovy/org/apache/jackrabbit/oak/console | oak-operations|
> | Backup | | oak-operations|
> | Restore | | oak-operations|
> | Debug | | oak-operations|
> | Check | | oak-operations|
> | Compact | | oak-operations|
> | Server | | oak-development|
> | Upgrade | | oak-upgrade|
> | Explore | |oak-development |
> | Primary | | oak-development|
> | Standby | | oak-development|
> | Help | | |
> | Checkpoints | | oak-operations|
> | Recovery | | oak-operations|
> | Repair | | oak-operations|
> | Tika | | |
> Modules left after the actions:
> **oak-development**
> Used to group and execute all the tools used during development.
> _deployed to maven:_ No.
> **oak-operations**
> Used to group and execute all the tools used normally in production 
> environment for tasks like maintenance & C.
> _deployed to maven:_ Yes.
> **oak-run**
> Will group what's left after the split.
> _deployed to maven:_ No.
> **oak-upgrade**
> Will group tools for upgrading/migrating from one repo/version to another
> _deployed to maven:_ yes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3763) EmptyNodeState.equals() broken

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052580#comment-15052580
 ] 

Michael Dürig commented on OAK-3763:


+1, looks good.

> EmptyNodeState.equals() broken
> --
>
> Key: OAK-3763
> URL: https://issues.apache.org/jira/browse/OAK-3763
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: core
>Affects Versions: 1.0, 1.2
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
> Fix For: 1.4
>
> Attachments: OAK-3763.patch
>
>
> EmptyNodeState.equals() returns incorrect results when the other node state 
> is not of type EmptyNodeState and the two states have differing exists() 
> flags.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3158) IAE when specifiying 2G cache for FileStore

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3158:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 resilience  (was: resilience)

> IAE when specifiying 2G cache for FileStore
> ---
>
> Key: OAK-3158
> URL: https://issues.apache.org/jira/browse/OAK-3158
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Thomas Mueller
>Priority: Minor
>  Labels: candidate_oak_1_0, candidate_oak_1_2, resilience
> Fix For: 1.3.6
>
>
> {{FileStore.newFileStore(dir).withCacheSize(2048)}} results in
> {noformat}Max memory must not be negative
> java.lang.IllegalArgumentException: Max memory must not be negative
>   at 
> org.apache.jackrabbit.oak.cache.CacheLIRS.setMaxMemory(CacheLIRS.java:464)
>   at org.apache.jackrabbit.oak.cache.CacheLIRS.(CacheLIRS.java:163)
>   at 
> org.apache.jackrabbit.oak.cache.CacheLIRS$Builder.build(CacheLIRS.java:1537)
>   at 
> org.apache.jackrabbit.oak.cache.CacheLIRS$Builder.build(CacheLIRS.java:1533)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.StringCache.(StringCache.java:52)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.(SegmentTracker.java:126)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.(FileStore.java:343)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.(FileStore.java:84)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore$Builder.create(FileStore.java:294)
> {noformat}
> There is an integer overflow cause by using ints instead of longs to specify 
> the cache size.
> [~tmueller], could you have a look?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3048) Enable lookup of OSGi configuration from framework first and component next

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3048:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> Enable lookup of OSGi configuration from framework first and component next
> ---
>
> Key: OAK-3048
> URL: https://issues.apache.org/jira/browse/OAK-3048
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Reporter: Francesco Mari
>Assignee: Davide Giannella
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.4
>
> Attachments: OAK-3048-01.patch, OAK-3048-02.patch
>
>
> The {{OsgiUtil}} class only allows looking up OSGi configuration from the 
> component context first and from the framework next. 
> Other components, like {{DocumentNodeStoreService}}, need a utility method 
> that implements the complementary strategy - framework first, component 
> context next.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3125) Skip compaction estimation if threshold is 0

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3125:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Skip compaction estimation if threshold is 0
> 
>
> Key: OAK-3125
> URL: https://issues.apache.org/jira/browse/OAK-3125
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core, segmentmk
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Minor
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.4
>
> Attachments: OAK-3125.patch
>
>
> as noted on OAK-3052, it would be good to have a way to skip the estimation 
> if needed  and this can be easily achieved with the config we already in 
> place for the threshold.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-1995) Improved SegmentNodeStore documentation

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-1995:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> Improved SegmentNodeStore documentation
> ---
>
> Key: OAK-1995
> URL: https://issues.apache.org/jira/browse/OAK-1995
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: doc, segmentmk
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
>Priority: Minor
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.11
>
>
> I'd like to have more documentation to understand the inner workings of the 
> SegmentNodeStore implementation.
> * Architecture level docs
> * Class level javadocs
> * Field javadocs
> * Method level javadocs
> * More complete file format docs
> Maybe it's possible to make a few things "more" private.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3138) OOME in NodeStateCopierTest

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3138:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> OOME in NodeStateCopierTest
> ---
>
> Key: OAK-3138
> URL: https://issues.apache.org/jira/browse/OAK-3138
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: upgrade
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.4
>
>
> {noformat}
> shouldNotDeleteExistingNodesIfMerged(org.apache.jackrabbit.oak.upgrade.nodestate.NodeStateCopierTest)
>   Time elapsed: 4.257 sec  <<< ERROR!
> java.lang.OutOfMemoryError: Java heap space
>   at 
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.clear(CacheLIRS.java:819)
>   at 
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.(CacheLIRS.java:774)
>   at 
> org.apache.jackrabbit.oak.cache.CacheLIRS.invalidateAll(CacheLIRS.java:186)
>   at org.apache.jackrabbit.oak.cache.CacheLIRS.(CacheLIRS.java:173)
>   at 
> org.apache.jackrabbit.oak.cache.CacheLIRS$Builder.build(CacheLIRS.java:1537)
>   at 
> org.apache.jackrabbit.oak.cache.CacheLIRS$Builder.build(CacheLIRS.java:1533)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.StringCache.(StringCache.java:55)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.(SegmentTracker.java:124)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.(SegmentTracker.java:144)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.memory.MemoryStore.(MemoryStore.java:45)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.memory.MemoryStore.(MemoryStore.java:62)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.(SegmentNodeStore.java:118)
>   at 
> org.apache.jackrabbit.oak.upgrade.util.NodeStateTestUtils.createNodeStoreWithContent(NodeStateTestUtils.java:49)
>   at 
> org.apache.jackrabbit.oak.upgrade.nodestate.NodeStateCopierTest.shouldNotDeleteExistingNodesIfMerged(NodeStateCopierTest.java:131)
> {noformat}
> This is most likely caused by the changes of caching introduced with 
> OAK-3055. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-2713) High memory usage of CompactionMap

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-2713:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc resilience  (was: 
compaction gc resilience)

> High memory usage of CompactionMap
> --
>
> Key: OAK-2713
> URL: https://issues.apache.org/jira/browse/OAK-2713
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc, 
> resilience
> Fix For: 1.3.0, 1.2.3, 1.0.15
>
>
> In environments with a lot of volatile content the {{CompactionMap}} can end 
> up eating a lot of memory. From 
> {{CompactionStrategyMBean#getCompactionMapStats}}:
> {noformat}
> [Estimated Weight: 317,5 MB, Records: 39500094, Segments: 36698], 
> [Estimated Weight: 316,4 MB, Records: 39374593, Segments: 36660], 
> [Estimated Weight: 315,4 MB, Records: 39253205, Segments: 36620], 
> [Estimated Weight: 315,1 MB, Records: 39221882, Segments: 36614], 
> [Estimated Weight: 314,9 MB, Records: 39195490, Segments: 36604], 
> [Estimated Weight: 315,0 MB, Records: 39182753, Segments: 36602], 
> [Estimated Weight: 360 B, Records: 0, Segments: 0],
> {noformat}
> This causes compaction to be skipped:
> {noformat}
> 2015-03-30:30.03.2015 02:00:00.038 *INFO* [] [TarMK compaction thread 
> [/foo/bar/crx-quickstart/repository/segmentstore], active since Mon Mar 30 
> 02:00:00 CEST 2015, previous max duration 3854982ms] 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore Not enough available 
> memory 5,5 GB, needed 6,3 GB, last merge delta 1,3 GB, so skipping compaction 
> for now
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3767) define index in subdirectories of oak:index

[Davide Giannella (JIRA)]

Davide Giannella created OAK-3767:
-

 Summary: define index in subdirectories of oak:index
 Key: OAK-3767
 URL: https://issues.apache.org/jira/browse/OAK-3767
 Project: Jackrabbit Oak
  Issue Type: Improvement
  Components: core, query
Reporter: Davide Giannella


Provide the ability to define indexes in subdirectories of an oak:index node. 
Something like

{noformat}
/oak:index
 -> oak
 -> apps
{noformat}

*question*. Shall we allow this only under root or under any arbitrary path as 
well? For example /content.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3737) Compactor should log revisions acting upon

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3737:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Compactor should log revisions acting upon
> --
>
> Key: OAK-3737
> URL: https://issues.apache.org/jira/browse/OAK-3737
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Blocker
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.12
>
>
> For post mortem analysis it would be helpful to have the revisions that where 
> involved in a compaction run. I.e. the revision that was compacted, the 
> revisions of the cycles (if any) and the revision that is ultimately applied?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3732) Offline compaction doesn't clean up unreferenced tar files

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3732:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 cleanup gc  (was: cleanup gc)

> Offline compaction doesn't clean up unreferenced tar files
> --
>
> Key: OAK-3732
> URL: https://issues.apache.org/jira/browse/OAK-3732
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, gc
> Fix For: 1.3.12
>
>
> This is a regression introduced with OAK-3329 where cleaning up unreferenced 
> tar files was taken out of {{FileStore#cleanup}}. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052672#comment-15052672
 ] 

Julian Reschke edited comment on OAK-3743 at 12/11/15 12:04 PM:


Right. Can you check whether the weirdness wrt test results ending in the wrong 
log files has gone, though?

Maybe we should try to reduce the amount of parallelism in the oak-jcr tests 
and see whether that changes things?


was (Author: reschke):
Right. Can you check whether the weirdness wrt test results ending in the wrong 
log files has gone, though?

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3766) Investigate and remove dependencies from oak-run

[Thomas Mueller (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052577#comment-15052577
 ] 

Thomas Mueller commented on OAK-3766:
-

And org.mapdb. It is used in the 1.0 and 1.2 branches of oak-core (for the 
DocumentNodeStore), but no longer seems to be needed in trunk.

> Investigate and remove dependencies from oak-run
> 
>
> Key: OAK-3766
> URL: https://issues.apache.org/jira/browse/OAK-3766
> Project: Jackrabbit Oak
>  Issue Type: Task
>Reporter: Davide Giannella
>
> While with OAK-3134 the effort is to split oak-run into smaller bundle to 
> ease deployment and maintenance this is to see if we can easily cut the size 
> of it by removing unused dependencies.
> *dependencies identified so far to investigate*
> - hadoop
> - solr
> - tika
> - jetty
> - zookeeper
> - H2 (SQL part)
> - JR remoting



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3529) NodeStore API should expose an Instance ID

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052592#comment-15052592
 ] 

Michael Dürig commented on OAK-3529:


bq. There's currently no other way to have access to the underlying NodeStore 
when we construct a repository in the new Jcr(...).

This is only true for the default case, where the builder falls back to a 
memory node store and an instance id wouldn't be available anyway. For the 
other cases users pass in a node store explicitly in some place and could grab 
the instance id there directly I guess. 

> NodeStore API should expose an Instance ID
> --
>
> Key: OAK-3529
> URL: https://issues.apache.org/jira/browse/OAK-3529
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.4
>
> Attachments: OAK-3529-1.patch, OAK-3529-2.patch, OAK-3529-3.patch, 
> OAK-3529-4.patch
>
>
> For better leveraging cluster oriented algorithm: discovery, atomic
> operations; it would be very helpful if the NodeStore could expose a
> unique instance id.
> This can be the same as a cluster ID.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3766) Investigate and remove dependencies from oak-run

[Thomas Mueller (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052618#comment-15052618
 ] 

Thomas Mueller commented on OAK-3766:
-

With this diff, the oak-run jar file is 30 instead of 50 MB. 

I changed Solr to scope "provided", [~teofili] is that the right way? Or maybe 
scope "test"? I couldn't find a way to exclude just Hadoop.

{noformat}
Index: oak-core/pom.xml
===
--- oak-core/pom.xml(revision 1718800)
+++ oak-core/pom.xml(working copy)
@@ -243,7 +243,7 @@
 
 
   com.h2database
-  h2
+  h2-mvstore
   ${h2.version}
   true
 
Index: oak-run/pom.xml
===
--- oak-run/pom.xml (revision 1718800)
+++ oak-run/pom.xml (working copy)
@@ -259,16 +259,19 @@
   org.apache.jackrabbit
   oak-solr-core
   ${project.version}
+  provided
 
 
   org.apache.solr
   solr-solrj
   ${lucene.version}
+  provided
 
 
   org.apache.solr
   solr-core
   ${lucene.version}
+  provided
   
 
   org.apache.lucene
@@ -288,7 +291,7 @@
 
 
   com.h2database
-  h2
+  h2-mvstore
   ${h2.version}
 
 
@@ -361,12 +364,6 @@
   provided
 
 
-  org.mapdb
-  mapdb
-  1.0.6
-  compile
-
-
   com.googlecode.json-simple
   json-simple
   1.1.1
@@ -466,8 +463,8 @@
   org.apache.jackrabbit
   oak-solr-osgi 
   ${project.version}
+  provided
 
-
   
 
   
Index: 
oak-run/src/main/java/org/apache/jackrabbit/oak/plugins/document/util/MapDBMapFactory.java
===
--- 
oak-run/src/main/java/org/apache/jackrabbit/oak/plugins/document/util/MapDBMapFactory.java
  (revision 1718800)
+++ 
oak-run/src/main/java/org/apache/jackrabbit/oak/plugins/document/util/MapDBMapFactory.java
  (working copy)
@@ -1,80 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements.  See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License.  You may obtain a copy of the License at
- *
- *  http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.jackrabbit.oak.plugins.document.util;
-
-import java.io.DataInput;
-import java.io.DataOutput;
-import java.io.IOException;
-import java.io.Serializable;
-import java.util.concurrent.atomic.AtomicInteger;
-
-import org.apache.jackrabbit.oak.plugins.document.Revision;
-import org.mapdb.BTreeMap;
-import org.mapdb.DB;
-import org.mapdb.DBMaker;
-import org.mapdb.Serializer;
-
-/**
- * A MapFactory backed by MapDB, which stores the map in a temporary file.
- */
-public class MapDBMapFactory extends MapFactory {
-
-private final AtomicInteger counter = new AtomicInteger();
-private final DB db;
-
-public MapDBMapFactory() {
-this.db = DBMaker.newTempFileDB()
-.deleteFilesAfterClose()
-.closeOnJvmShutdown()
-.transactionDisable()
-.asyncWriteEnable()
-.make();
-}
-
-@Override
-public BTreeMap create() {
-return db.createTreeMap(String.valueOf(counter.incrementAndGet()))
-.valueSerializer(new RevisionSerializer())
-.counterEnable()
-.makeStringMap();
-}
-
-private static class RevisionSerializer implements Serializer,
-Serializable {
-private static final long serialVersionUID = 8648365575103098316L;
-private int size = 8 + 4 + 4 + 1;
-public void serialize(DataOutput o, Revision r) throws IOException {
-o.writeLong(r.getTimestamp());
-o.writeInt(r.getCounter());
-o.writeInt(r.getClusterId());
-o.writeBoolean(r.isBranch());
-
-}
-
-public Revision deserialize(DataInput i, int available) throws 
IOException {
-return new Revision(
-i.readLong(), //timestamp
-i.readInt(),  //counter
-i.readInt(),  //clusterId
-i.readBoolean()); //branch
-}
-
-public int fixedSize() {
-return size;
-

[jira] [Updated] (OAK-1446) Offline tool to repair TarMK

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-1446:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 production resilience tools  
(was: production resilience tools)

> Offline tool to repair TarMK
> 
>
> Key: OAK-1446
> URL: https://issues.apache.org/jira/browse/OAK-1446
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Marth
>Assignee: Jukka Zitting
>  Labels: candidate_oak_1_0, candidate_oak_1_2, production, 
> resilience, tools
>
> We should have a tool to inspect and repair TarMK files in case the 
> repository does not come up due to a corruption in the tar files.
> Ideally, the tool could be pointed to an existing backup and use the backup 
> to fix broken binaries (that might have been erroneously been deleted by the 
> DS GC).
> Once we have the tool, we could automatically run it after backups and on the 
> failover instance (OAK-1161)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-2581) Metatype info for SegmentNodeStoreService

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-2581:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> Metatype info for SegmentNodeStoreService
> -
>
> Key: OAK-2581
> URL: https://issues.apache.org/jira/browse/OAK-2581
> Project: Jackrabbit Oak
>  Issue Type: Sub-task
>  Components: segmentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.1.8
>
> Attachments: OAK-2581-1.patch, OAK-2581.patch, 
> segment-nodestore-config.png
>
>
> Sub task for adding metatype info for {{SegmentNodeStoreService}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052672#comment-15052672
 ] 

Julian Reschke commented on OAK-3743:
-

Right. Can you check whether the weirdness wrt test results ending in the wrong 
log files has gone, though?

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3348) Cross gc sessions might introduce references to pre-compacted segments

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3348:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 cleanup compaction gc  (was: 
cleanup compaction gc)

> Cross gc sessions might introduce references to pre-compacted segments
> --
>
> Key: OAK-3348
> URL: https://issues.apache.org/jira/browse/OAK-3348
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, 
> compaction, gc
> Fix For: 1.4
>
> Attachments: OAK-3348-1.patch, OAK-3348-2.patch, OAK-3348.patch, 
> cross-gc-refs.pdf, image.png
>
>
> I suspect that certain write operations during compaction can cause 
> references from compacted segments to pre-compacted ones. This would 
> effectively prevent the pre-compacted segments from getting evicted in 
> subsequent cleanup phases. 
> The scenario is as follows:
> * A session is opened and a lot of content is written to it such that the 
> update limit is exceeded. This causes the changes to be written to disk. 
> * Revision gc runs causing a new, compacted root node state to be written to 
> disk.
> * The session saves its changes. This causes rebasing of its changes onto the 
> current root (the compacted one). At this point any node that has been added 
> will be added again in the sub-tree rooted at the current root. Such nodes 
> however might have been written to disk *before* revision gc ran and might 
> thus be contained in pre-compacted segments. As I suspect the node-add 
> operation in the rebasing process *not* to create a deep copy of such nodes 
> but to rather create a *reference* to them, a reference to a pre-compacted 
> segment is introduced here. 
> Going forward we need to validate above hypothesis, assess its impact if 
> necessary come up with a solution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3668) Potential test failure: CompactionAndCleanupIT#testMixedSegments

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3668:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Potential test failure: CompactionAndCleanupIT#testMixedSegments
> 
>
> Key: OAK-3668
> URL: https://issues.apache.org/jira/browse/OAK-3668
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.4
>
> Attachments: OAK-3668.patch
>
>
> {{CompactionAndCleanupIT#testMixedSegments}} might fail under some 
> circumstances. It can be certainly be made to fail by increasing concurrency. 
> I suspect this to be caused by OAK-3348. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3052) Make compaction gain estimate threshold configurable

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3052:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Make compaction gain estimate threshold configurable
> 
>
> Key: OAK-3052
> URL: https://issues.apache.org/jira/browse/OAK-3052
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Minor
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.3
>
> Attachments: OAK-3052-v2.patch, OAK-3052.patch
>
>
> Currently compaction is skipped if the compaction gain estimator determines 
> that less than 10% of space could be reclaimed. Instead of relying on a hard 
> coded value of 10% we should make this configurable. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3529) NodeStore API should expose an Instance ID

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052661#comment-15052661
 ] 

Michael Dürig commented on OAK-3529:


+1 for separating the concern re. {{Clusterable}}. Checking the passed node 
store for {{Clusterable}} should unblock the atomic counter. Generalising 
{{Clusterable}} warrants a broader discussion IMO.

Re. the patch, there is a problem in the order of initialisation: the {{store}} 
member is used before it is initialised in {{Jcr(Oak)}}. 

> NodeStore API should expose an Instance ID
> --
>
> Key: OAK-3529
> URL: https://issues.apache.org/jira/browse/OAK-3529
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.4
>
> Attachments: OAK-3529-1.patch, OAK-3529-2.patch, OAK-3529-3.patch, 
> OAK-3529-4.patch, OAK-3529-5.patch
>
>
> For better leveraging cluster oriented algorithm: discovery, atomic
> operations; it would be very helpful if the NodeStore could expose a
> unique instance id.
> This can be the same as a cluster ID.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-2714) Test failures on Jenkins

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-2714:
---
Description: 
This issue is for tracking test failures seen at our Jenkins instance that 
might yet be transient. Once a failure happens too often we should remove it 
here and create a dedicated issue for it. 

|| Test 
  || Builds || Fixture  || JVM ||
| org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopierTest.reuseLocalDir  
   | 81  | DOCUMENT_RDB | 1.7   |
| org.apache.jackrabbit.oak.jcr.OrderedIndexIT.oak2035  
   | 76, 128 | SEGMENT_MK , DOCUMENT_RDB  | 1.6   |
| org.apache.jackrabbit.oak.plugins.segment.standby.StandbyTestIT.testSyncLoop  
   | 64  | ?| ? |
| 
org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest.testRDBDocumentStore_CustomBlobStore
 | 52, 181, 399 |  SEGMENT_MK, DOCUMENT_NS | 1.7 |
| org.apache.jackrabbit.oak.run.osgi.JsonConfigRepFactoryTest.testRepositoryTar 
   | 41  | ?| ? |
| 
org.apache.jackrabbit.test.api.observation.PropertyAddedTest.testMultiPropertyAdded
  | 29  | ?| ? |
| org.apache.jackrabbit.oak.plugins.segment.HeavyWriteIT.heavyWrite 
   | 35  | SEGMENT_MK   | ? |
| 
org.apache.jackrabbit.oak.jcr.query.QueryPlanTest.propertyIndexVersusNodeTypeIndex
 | 90 | DOCUMENT_RDB | 1.6 |
| org.apache.jackrabbit.oak.jcr.observation.ObservationTest.removeSubtreeFilter 
| 94 | DOCUMENT_RDB | 1.6 |
| org.apache.jackrabbit.oak.jcr.observation.ObservationTest.disjunctPaths | 
121, 157, 396 | DOCUMENT_RDB | 1.6, 1.7, 1.8 |
| org.apache.jackrabbit.oak.jcr.ConcurrentFileOperationsTest.concurrent | 110, 
382 | DOCUMENT_RDB | 1.6 |
| org.apache.jackrabbit.oak.jcr.MoveRemoveTest.removeExistingNode | 115 | 
DOCUMENT_RDB | 1.7 |
| org.apache.jackrabbit.oak.jcr.RepositoryTest.addEmptyMultiValue | 115 | 
DOCUMENT_RDB | 1.7 |
| org.apache.jackrabbit.oak.stats.ClockTest.testClockDriftFast | 115, 142 | 
SEGMENT_MK, DOCUMENT_NS | 1.6, 1.8 |
| org.apache.jackrabbit.oak.plugins.index.solr.query.SolrQueryIndexTest | 148, 
151, 490 | SEGMENT_MK, DOCUMENT_NS, DOCUMENT_RDB | 1.6, 1.7, 1.8 |
| org.apache.jackrabbit.oak.jcr.observation.ObservationTest.testReorder | 155 | 
DOCUMENT_RDB | 1.8 |
| org.apache.jackrabbit.oak.osgi.OSGiIT.bundleStates | 163 | SEGMENT_MK | 1.6 |
| org.apache.jackrabbit.oak.jcr.query.SuggestTest | 171 | SEGMENT_MK | 1.8 |
| org.apache.jackrabbit.oak.jcr.nodetype.NodeTypeTest.updateNodeType | 243, 400 
| DOCUMENT_RDB | 1.6, 1.8 |
| org.apache.jackrabbit.oak.jcr.query.QueryPlanTest.nodeType | 272 | 
DOCUMENT_RDB | 1.8 |
| org.apache.jackrabbit.oak.jcr.observation.ObservationTesttestMove | 308 | 
DOCUMENT_RDB | 1.6 |
| 
org.apache.jackrabbit.oak.jcr.version.VersionablePathNodeStoreTest.testVersionablePaths
 | 361 | DOCUMENT_RDB | 1.7 |
| org.apache.jackrabbit.oak.plugins.document.DocumentDiscoveryLiteServiceTest | 
361, 608 | DOCUMENT_NS, SEGMENT_MK | 1.7, 1.8 |
| org.apache.jackrabbit.oak.jcr.ConcurrentAddIT.addNodesSameParent | 427, 428 | 
DOCUMENT_NS, SEGMENT_MK | 1.7 |
| Build crashes: malloc(): memory corruption | 477 | DOCUMENT_NS | 1.6 |
| org.apache.jackrabbit.oak.upgrade.cli.SegmentToJdbcTest.validateMigration | 
486 | DOCUMENT_NS | 1.7| 
| org.apache.jackrabbit.j2ee.TomcatIT.testTomcat | 489, 493, 597 | DOCUMENT_NS, 
SEGMENT_MK | 1.7 | 
| org.apache.jackrabbit.oak.plugins.index.solr.index.SolrIndexEditorTest | 490 
| DOCUMENT_RDB | 1.7 |
| 
org.apache.jackrabbit.oak.plugins.index.solr.server.EmbeddedSolrServerProviderTest.testEmbeddedSolrServerInitialization
 | 490 | DOCUMENT_RDB | 1.7 |
| 
org.apache.jackrabbit.oak.run.osgi.PropertyIndexReindexingTest.propertyIndexState
 | 492 | DOCUMENT_NS | 1.6 |
| org.apache.jackrabbit.j2ee.TomcatIT | 589 | SEGMENT_MK | 1.8 |




  was:
This issue is for tracking test failures seen at our Jenkins instance that 
might yet be transient. Once a failure happens too often we should remove it 
here and create a dedicated issue for it. 

|| Test 
  || Builds || Fixture  || JVM ||
| org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopierTest.reuseLocalDir  
   | 81  | DOCUMENT_RDB | 1.7   |
| org.apache.jackrabbit.oak.jcr.OrderedIndexIT.oak2035  
   | 76, 128 | SEGMENT_MK , DOCUMENT_RDB  | 1.6   |
| org.apache.jackrabbit.oak.plugins.segment.standby.StandbyTestIT.testSyncLoop  
   | 64  | ?| ? |
| 
org.apache.jackrabbit.oak.run.osgi.DocumentNodeStoreConfigTest.testRDBDocumentStore_CustomBlobStore
 | 52, 181, 399 |  SEGMENT_MK, DOCUMENT_NS | 1.7 |
| 

[jira] [Updated] (OAK-3179) Deadlock between persisted compaction map and cleanup

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3179:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 cleanup compaction gc  (was: 
cleanup compaction gc)

> Deadlock between persisted compaction map and cleanup
> -
>
> Key: OAK-3179
> URL: https://issues.apache.org/jira/browse/OAK-3179
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
>  Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, 
> compaction, gc
> Fix For: 1.3.4
>
>
> Just seen this deadlock while running {{SegmentCompactionIT}}:
> {noformat}
> "TarMK flush thread [target/SegmentCompactionIT3250704011919039778dir], 
> active since Wed Aug 05 09:25:57 GMT+00:00 2015, previous max duration 
> 2325ms" daemon prio=10 tid=0x7f5674872800 nid=0x5dc8 waiting for monitor 
> entry [0x7f5666a0]
>java.lang.Thread.State: BLOCKED (on object monitor)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:145)
>   - waiting to lock <0x000707fc7fe8> (a 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:154)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:186)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:186)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.compress(PersistedCompactionMap.java:204)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.remove(PersistedCompactionMap.java:155)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.CompactionMap.remove(CompactionMap.java:108)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.cleanup(FileStore.java:694)
>   - locked <0x00070017b330> (a 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.flush(FileStore.java:628)
>   - locked <0x00070017b330> (a 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
>   - locked <0x0007000aed60> (a 
> java.util.concurrent.atomic.AtomicReference)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore$1.run(FileStore.java:413)
>   at java.lang.Thread.run(Thread.java:745)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.BackgroundThread.run(BackgroundThread.java:70)
> "pool-1-thread-34" prio=10 tid=0x7f55ec002800 nid=0x5dea waiting for 
> monitor entry [0x7f56648de000]
>java.lang.Thread.State: BLOCKED (on object monitor)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.readSegment(FileStore.java:904)
>   - waiting to lock <0x00070017b330> (a 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.readSegment(SegmentTracker.java:210)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:149)
>   - locked <0x000707fc7fe8> (a 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:400)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:215)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:186)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntry(MapRecord.java:186)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.get(PersistedCompactionMap.java:121)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.PersistedCompactionMap.get(PersistedCompactionMap.java:103)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.CompactionMap.get(CompactionMap.java:93)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.uncompact(SegmentWriter.java:1074)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1098)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1154)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1154)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1154)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
>   at 
> 

[jira] [Updated] (OAK-3051) Improve compaction gain estimation logging for the case where there are no tar readers

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3051:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Improve compaction gain estimation logging for the case where there are no 
> tar readers
> --
>
> Key: OAK-3051
> URL: https://issues.apache.org/jira/browse/OAK-3051
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Minor
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.3
>
>
> When compaction is started on a new and sufficiently small repository such 
> that there is yet no tar reader the compaction gain estimator logs the 
> somewhat confusing message {{gain is 0% (0/0) or (0 B/0 B)}}. 
> We should improve the logging for this case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3095) Add eviction listener to LIRS cache

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3095:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> Add eviction listener to LIRS cache
> ---
>
> Key: OAK-3095
> URL: https://issues.apache.org/jira/browse/OAK-3095
> Project: Jackrabbit Oak
>  Issue Type: New Feature
>  Components: core
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.3
>
> Attachments: OAK-3095-2.patch, OAK-3095.patch
>
>
> For OAK-3055 I need to be able to track items that are evicted from 
> {{CacheLIRS}}. I thus suggest to implement a listener for evicted items. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3177) Compaction slow on repository with continuous writes

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3177:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Compaction slow on repository with continuous writes
> 
>
> Key: OAK-3177
> URL: https://issues.apache.org/jira/browse/OAK-3177
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.4
>
> Attachments: OAK-3177.patch, OAK-3177.png
>
>
> OAK-2734 introduced retry cycles and the option to force compaction when all 
> cycles fail. However OAK-2192 introduced a performance regression: each 
> compaction cycle takes in the order of the size of the repository to complete 
> instead of in the order of the number of remaining changes to compact. This 
> is caused by comparing compacted with pre-compacted node states, which is 
> necessary to avoid mixed segments (aka OAK-2192). To fix the performance 
> regression I propose to pass the compactor an additional node state (the 
> 'onto' state). The diff would then be calculated across the pre compacted 
> states, which performs in the order of number of changes. The changes would 
> then be applied to the 'onto' state, which is a compacted state to avoid 
> mixed segments. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3168) SegmentCache flushes Segment on update

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3168:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> SegmentCache flushes Segment on update
> --
>
> Key: OAK-3168
> URL: https://issues.apache.org/jira/browse/OAK-3168
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.4
>
>
> The SegmentCache currently uses the cache eviction call to remove the Segment 
> instance from memory to help keep the cache memory requirements under control 
> [0].
> What I've noticed though, is that for a cache update (existing key) there 
> will also be an eviction call happening, which results in a lot of extra IO 
> pressure on the SegmentStore which not only is not able to cache the segment, 
> but is forced to reload it multiple times as the reference gets nullified 
> after each load.
> This comes from the sampling behavior of the SegmentId: it will not hit the 
> cache each time it needs to load a new Segment, but rather load it from IO 
> and (re)place it in the cache, based on a sampling rate [1].
> Now I see 2 options:
>  * change the cache code to _not_ call the eviction callback on updates (or 
> allow disabling this call on updates)
>  * change the SegmentTracker code to add the value to the cache only if it's 
> not there as Segments are immutable, so no harm done.
> Raised this issue offline with [~tmueller], [~mduerig] first and as I 
> understand [~mduerig] is in favor of option one, while [~tmueller] proposed 
> that the Lirs cache impl should be inline with what the guava cache does, and 
> depending on that we could choose the right fix here. 
> Hope this covers everything.
> [0] 
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentTracker.java#L133
> [1] 
> https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentId.java#L135



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-2384) SegmentNotFoundException when keeping JCR Value references

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-2384:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 gc  (was: gc)

> SegmentNotFoundException when keeping JCR Value references
> --
>
> Key: OAK-2384
> URL: https://issues.apache.org/jira/browse/OAK-2384
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: core
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
>  Labels: candidate_oak_1_0, candidate_oak_1_2, gc
> Fix For: 1.0.12, 1.1.8
>
> Attachments: org.apache.sling.jcr.resource-2.4.0.jar
>
>
> With OAK-2192 revision gc started to remove segments older than a certain 
> threshold. The underlying assumption was that old sessions would call refresh 
> (i.e. auto refresh) anyway once they become active again. However, it turns 
> out that refreshing a sessions does not affect JCR values as those are 
> directly tied to the underlying record. Accessing those values after its 
> segment has been gc'ed results in a {{SegmentNotFoundException}}. 
> Keeping reference to JCR values is an important use case for Sling's 
> {{JcrPropertyMap}}, which is widely used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3109) OOME in tarkmk standby tests

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3109:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> OOME in tarkmk standby tests
> 
>
> Key: OAK-3109
> URL: https://issues.apache.org/jira/browse/OAK-3109
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: tarmk-standby
>Reporter: Michael Dürig
>Assignee: Alex Parvulescu
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.4
>
>
> Running the tarmk-standby ITs results in an OOME when running with the new 
> string cache from OAK-3007. Looking at the heap dump I see many 
> {{SegmentTracker}} instances around, which leads me to think that the test 
> setup might have a leak somewhere. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3055) Improve segment cache in SegmentTracker

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3055:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 doc-impacting resilience 
scalability  (was: doc-impacting resilience scalability)

> Improve segment cache in SegmentTracker
> ---
>
> Key: OAK-3055
> URL: https://issues.apache.org/jira/browse/OAK-3055
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, doc-impacting, 
> resilience, scalability
> Fix For: 1.3.3
>
> Attachments: OAK-3055.patch
>
>
> The hand crafted segment cache in {{SegmentTracker}} is prone to lock 
> contentions in concurrent access scenarios. As {{SegmentNodeStore#merge}} 
> might also end up acquiring this lock while holding the commit semaphore the 
> situation can easily lead to many threads being blocked on the commit 
> semaphore. The {{SegmentTracker}} cache doesn't differentiate between read 
> and write access, which means that reader threads can block writer threads. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3766) Investigate and remove dependencies from oak-run

[Alex Parvulescu (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052664#comment-15052664
 ] 

Alex Parvulescu commented on OAK-3766:
--

I would like to remove {{oak-tarmk-standby}} support from oak-run. it was there 
to showcase the project, but now it's not needed anymore, or if anyone needs 
it, I would be willing to provide the demo as a part of the standby project 
itself.

> Investigate and remove dependencies from oak-run
> 
>
> Key: OAK-3766
> URL: https://issues.apache.org/jira/browse/OAK-3766
> Project: Jackrabbit Oak
>  Issue Type: Task
>Reporter: Davide Giannella
>
> While with OAK-3134 the effort is to split oak-run into smaller bundle to 
> ease deployment and maintenance this is to see if we can easily cut the size 
> of it by removing unused dependencies.
> *dependencies identified so far to investigate*
> - hadoop
> - solr
> - tika
> - jetty
> - zookeeper
> - H2 (SQL part)
> - JR remoting



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3743) Build time out after 90 mins on Jenkins

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3743:
---
Description: 
Every 2nd build or so is aborted after a time out (90 mins):

{noformat}
Build timed out (after 90 minutes). Marking the build as aborted.
Build was aborted
[FINDBUGS] Skipping publisher since build result is ABORTED
Recording test results
Finished: ABORTED
{noformat}

See  e.g. 
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console

Interestingly when successful, the build takes about 30mins, so way below the 
90 min. timeout. 

Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 

  was:
Every 2nd build or so is aborted after a time out (90 mins):

{noformat}
Build timed out (after 90 minutes). Marking the build as aborted.
Build was aborted
[FINDBUGS] Skipping publisher since build result is ABORTED
Recording test results
Finished: ABORTED
{noformat}

See  e.g. 
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console

Interestingly when successful, the build takes about 30mins, so way below the 
90 min. timeout. 

Also seen at builds 587, 590, 591, 593, 595, 597, 598


> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3506) Uniformization of compaction log messages

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3506:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Uniformization of compaction log messages
> -
>
> Key: OAK-3506
> URL: https://issues.apache.org/jira/browse/OAK-3506
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Reporter: Valentin Olteanu
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.9
>
> Attachments: compaction_logs.git.patch
>
>
> The logs generated during different phases of tar garbage collection 
> (compaction) are currently quite heterogenous and difficult to grep/parse.
> I propose with the attached patch to uniformize these logs, changing the 
> following:
> # all logs start with the prefix {{TarMK GargabeCollection \{\}#:}}
> # different phases of garbage collection are easier to identify by the first 
> word after prefix, e.g. estimation, compaction, cleanup
> # all values are also printed in a standard unit, with the following format: 
> {{ ()}}. This makes extraction of 
> information much easier.
> # messages corresponding to the same cycle (run) can be grouped by including 
> the runId in the prefix.
> Note1: I don't have enough visibility, but the changes might impact any 
> system relying on the old format. Yet, I've seen they have changed before so 
> this might not be a real concern.
> Note2: the runId is implemented as a static variable, which is reset every 
> time the class is reloaded (e.g. at restart), so it is unique only during one 
> run.
> Below you can find an excerpt of old logs and new logs to compare:
> NEW:
> {code}
> 12.10.2015 16:11:56.705 *INFO* [TarMK compaction thread 
> [/Users/volteanu/workspace/test/qp/quickstart-author-4502/crx-quickstart/repository/segmentstore],
>  active since Mon Oct 12 16:11:56 CEST 2015, previous max duration 0ms] 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK 
> GarbageCollection #1: started
> 12.10.2015 16:11:56.707 *INFO* [TarMK compaction thread 
> [/Users/volteanu/workspace/test/qp/quickstart-author-4502/crx-quickstart/repository/segmentstore],
>  active since Mon Oct 12 16:11:56 CEST 2015, previous max duration 0ms] 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK 
> GarbageCollection #1: estimation started
> 12.10.2015 16:11:59.275 *INFO* [TarMK compaction thread 
> [/Users/volteanu/workspace/test/qp/quickstart-author-4502/crx-quickstart/repository/segmentstore],
>  active since Mon Oct 12 16:11:56 CEST 2015, previous max duration 0ms] 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK 
> GarbageCollection #1: estimation completed in 2.569 s (2567 ms). Gain is 16% 
> or 1.1 GB/1.3 GB (1062364160/1269737472 bytes), so running compaction
> 12.10.2015 16:11:59.275 *INFO* [TarMK compaction thread 
> [/Users/volteanu/workspace/test/qp/quickstart-author-4502/crx-quickstart/repository/segmentstore],
>  active since Mon Oct 12 16:11:56 CEST 2015, previous max duration 0ms] 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK 
> GarbageCollection #1: compaction started, 
> strategy=CompactionStrategy{paused=false, cloneBinaries=false, 
> cleanupType=CLEAN_OLD, olderThan=3600, memoryThreshold=5, 
> persistedCompactionMap=true, retryCount=5, forceAfterFail=true, 
> compactionStart=1444659116706, offlineCompaction=false}
> 12.10.2015 16:12:05.839 *INFO* [TarMK compaction thread 
> [/Users/volteanu/workspace/test/qp/quickstart-author-4502/crx-quickstart/repository/segmentstore],
>  active since Mon Oct 12 16:11:56 CEST 2015, previous max duration 0ms] 
> org.apache.jackrabbit.oak.plugins.segment.Compactor Finished compaction: 
> 420022 nodes, 772259 properties, 20544 binaries.
> 12.10.2015 16:12:07.459 *INFO* [TarMK compaction thread 
> [/Users/volteanu/workspace/test/qp/quickstart-author-4502/crx-quickstart/repository/segmentstore],
>  active since Mon Oct 12 16:11:56 CEST 2015, previous max duration 0ms] 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK 
> GarbageCollection #1: compaction completed in 8.184 s (8183 ms), after 0 
> cycles
> 12.10.2015 16:12:11.912 *INFO* [TarMK flush thread 
> [/Users/volteanu/workspace/test/qp/quickstart-author-4502/crx-quickstart/repository/segmentstore],
>  active since Mon Oct 12 16:12:11 CEST 2015, previous max duration 10ms] 
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK 
> GarbageCollection #1: cleanup started. Current repository size is 1.4 GB 
> (1368899584 bytes)
> 12.10.2015 16:12:12.368 *INFO* [TarMK flush thread 
> [/Users/volteanu/workspace/test/qp/quickstart-author-4502/crx-quickstart/repository/segmentstore],
>  active since Mon Oct 

[jira] [Updated] (OAK-2870) Introduce a SegmentNodeStoreBuilder to help wire a SegmentNodeStore

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-2870:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> Introduce a SegmentNodeStoreBuilder to help wire a SegmentNodeStore
> ---
>
> Key: OAK-2870
> URL: https://issues.apache.org/jira/browse/OAK-2870
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core, segmentmk
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.0
>
> Attachments: OAK-2870.patch
>
>
> Exposing the SegmentNodeStore for tests outside oak-core is quite tricky as 
> you need to access some compaction related methods which are basically 
> private and it doesn't make much sense in making them public. 
> So I'm proposing introducing a builder to help wire in a SegmentNodeStore if 
> needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3330) FileStore lock contention with concurrent writers

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3330:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction  (was: compaction)

> FileStore lock contention with concurrent writers
> -
>
> Key: OAK-3330
> URL: https://issues.apache.org/jira/browse/OAK-3330
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction
> Fix For: 1.3.9
>
> Attachments: OAK-3330.patch
>
>
> Concurrently writing to the file store can lead to a sever lock contention in 
> {{FileStore#readSegment}}. That method searches the current {{TarWriter}} 
> instance for the segment once it could not be found in any of the 
> {{TarReader}} instances. This is the point where synchronizes on the 
> {{FileStore}} instance, which leads to  the contention. 
> The effect is only observable once the segment cache becomes full and reads 
> actually need to go to the file store. Thus a possible improvement could be 
> to pin segments from the current tar writer to the cache. Alternatively we 
> could try to ease locking by employing read/write locks where possible. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3501) PersistedCompactionMap could release reference to records early

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3501:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> PersistedCompactionMap could release reference to records early
> ---
>
> Key: OAK-3501
> URL: https://issues.apache.org/jira/browse/OAK-3501
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>Priority: Minor
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.8
>
> Attachments: OAK-3501-v2.patch, OAK-3501.patch
>
>
> Whenever a PersistedCompactionMap becomes empty it will be eventually dropped 
> from the compaction maps chain. this will happen on the next compaction 
> cycle, which happens post-cleanup. so we're potentially keeping a reference 
> to some collectable garbage for up to 2 cycles.
> I'd like to propose a patch that allows for eagerly nullifying the reference 
> to the records, making this interval shorter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3502) Improve logging during cleanup

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3502:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 cleanup gc  (was: cleanup gc)

> Improve logging during cleanup
> --
>
> Key: OAK-3502
> URL: https://issues.apache.org/jira/browse/OAK-3502
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, gc
> Fix For: 1.3.8
>
>
> We should have better logging during cleanup. E.g. why a file has been 
> skipped / cleaned etc. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3317) ConcurrentModificationException when running SegmentOverflowExceptionIT

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3317:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> ConcurrentModificationException when running SegmentOverflowExceptionIT
> ---
>
> Key: OAK-3317
> URL: https://issues.apache.org/jira/browse/OAK-3317
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.5
>
>
> {noformat}
> Running org.apache.jackrabbit.oak.plugins.segment.SegmentOverflowExceptionIT
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2,426.873 sec 
> <<< FAILURE!
> run(org.apache.jackrabbit.oak.plugins.segment.SegmentOverflowExceptionIT)  
> Time elapsed: 2,426.373 sec  <<< ERROR!
> java.util.ConcurrentModificationException
>   at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)
>   at java.util.ArrayList$Itr.next(ArrayList.java:831)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.CompactionMap.wasCompactedTo(CompactionMap.java:60)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.Record.wasCompactedTo(Record.java:64)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.equals(SegmentBlob.java:213)
>   at com.google.common.base.Objects.equal(Objects.java:55)
>   at 
> org.apache.jackrabbit.oak.plugins.memory.AbstractPropertyState.equal(AbstractPropertyState.java:53)
>   at 
> org.apache.jackrabbit.oak.plugins.memory.AbstractPropertyState.equals(AbstractPropertyState.java:90)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1176)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:100)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore$Commit.(SegmentNodeStore.java:418)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStore.merge(SegmentNodeStore.java:204)
>   at 
> org.apache.jackrabbit.oak.plugins.segment.SegmentOverflowExceptionIT.run(SegmentOverflowExceptionIT.java:130)
> {noformat}
> This is caused by concurrently accessing the underlying list of maps in 
> {{CompactionMap#remove}}. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3481) CompationMapTest does not close file store

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3481:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 cleanup compaction gc  (was: 
cleanup compaction gc)

> CompationMapTest does not close file store
> --
>
> Key: OAK-3481
> URL: https://issues.apache.org/jira/browse/OAK-3481
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: core, segmentmk
>Affects Versions: 1.3.4
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
>Priority: Minor
>  Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, 
> compaction, gc
> Fix For: 1.3.8
>
>
> Each test in the class retains about 16MB of heap and subsequent tests may 
> fail with an OOME. E.g. see recent build failure on travis: 
> https://travis-ci.org/apache/jackrabbit-oak/builds/83848567



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3329) TarMK cleanup blocks writers

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3329:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 cleanup gc  (was: cleanup gc)

> TarMK cleanup blocks writers
> 
>
> Key: OAK-3329
> URL: https://issues.apache.org/jira/browse/OAK-3329
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, gc
> Fix For: 1.3.8
>
> Attachments: OAK-3329.patch, base.png, patched.png
>
>
> TarMK cleanup exclusively locks the {{FileStore}}, which causes concurrent 
> writers to block until cleanup finished. Initially cleanup was expected to be 
> reasonably fast, however I have seen it taking dozens of minutes under 
> certain circumstances (most likely many tar files with many small segments, 
> aka OAK-2896).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3172) Unreleased closed sessions can keep a root reference from getting collected

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3172:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Unreleased closed sessions can keep a root reference from getting collected
> ---
>
> Key: OAK-3172
> URL: https://issues.apache.org/jira/browse/OAK-3172
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Alex Parvulescu
>Assignee: Alex Parvulescu
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.8
>
> Attachments: OAK-3172.patch
>
>
> It looks like even if a component logs out a session, but keeps a reference 
> to it around this will still prevent GC from running, as the session will 
> wrap a _root_ reference pointing to the moment/revision when the session was 
> last accessed.
> Extract from jvisualvm:
> {code}
> this - value: org.apache.jackrabbit.oak.plugins.segment.SegmentId #505
>  <- [106] - class: org.apache.jackrabbit.oak.plugins.segment.SegmentId[], 
> value: org.apache.jackrabbit.oak.plugins.segment.SegmentId #505
>   <- refids - class: org.apache.jackrabbit.oak.plugins.segment.Segment, 
> value: org.apache.jackrabbit.oak.plugins.segment.SegmentId[] #67 (120 items)
><- segment - class: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId, value: 
> org.apache.jackrabbit.oak.plugins.segment.Segment #81
> <- [124] - class: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId[], value: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId #496
>  <- refids - class: 
> org.apache.jackrabbit.oak.plugins.segment.Segment, value: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId[] #17 (204 items)
>   <- segment - class: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId, value: 
> org.apache.jackrabbit.oak.plugins.segment.Segment #17
><- segmentId - class: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState, value: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentId #551
> <- base - class: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder, value: 
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState #5557
>  <- builder - class: org.apache.jackrabbit.oak.core.MutableRoot, 
> value: org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder #5652
>   <- root - class: 
> org.apache.jackrabbit.oak.jcr.repository.RepositoryImpl$1, value: 
> org.apache.jackrabbit.oak.core.MutableRoot #151
><- sd - class: 
> com.adobe.granite.repository.impl.CRX3SessionImpl, value: 
> org.apache.jackrabbit.oak.jcr.repository.RepositoryImpl$1 #117
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3511) Test failure: CompactionMapTest.removeSome

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3511:
---
Labels: candidate_oak_1_0 candidate_oak_1_2 compaction gc  (was: compaction 
gc)

> Test failure: CompactionMapTest.removeSome
> --
>
> Key: OAK-3511
> URL: https://issues.apache.org/jira/browse/OAK-3511
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2, compaction, gc
> Fix For: 1.3.9
>
>
> Said test fails sporadically:
> {noformat}
> at org.junit.Assert.assertNull(Assert.java:562)
> at 
> org.apache.jackrabbit.oak.plugins.segment.CompactionMapTest.removeSome(CompactionMapTest.java:156)
> {noformat}
> This is a regression introduced with OAK-3501: the {{recent}} map gets not 
> cleared when {{segmentIdMap}} is empty. This can happen when a recent key is 
> removed again while there are no other changes. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3765) Parallelized test runner does not wait for test completion

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052776#comment-15052776
 ] 

Julian Reschke commented on OAK-3765:
-

That's an interesting idea. We'd need to think about interval, log level and 
format...

> Parallelized test runner does not wait for test completion
> --
>
> Key: OAK-3765
> URL: https://issues.apache.org/jira/browse/OAK-3765
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: jcr
>Affects Versions: 1.2.9, 1.0.25, 1.3.12
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Fix For: 1.3.13, 1.2.10
>
>
> As analyzed by [~tomek.rekawek] in OAK-3743:
> Quote from the mailing list:
> {quote}
> On a “slow” machine, in the surefire logs for the AtomicCounterTest (which 
> takes 63 sec while it should 3 sec), following test case appears:
> {noformat}
>classname="org.apache.jackrabbit.oak.jcr.RepositoryTest" 
> name="importWithRegisteredType[RDBDocumentStore on 
> jdbc:derby:oaktest\;create=true]"/>
> {noformat}
> It’s a test case from a completely different class. I downloaded all the 
> surefire reports and it seems that the test cases from the RepositoryTest 
> class are spread across CRUDTest, ConflictResolutionTest, ObservationTest and 
> others. On the “fast” machines the problem doesn’t exist and all 
> RepositoryTest methods are invoked within their own test case.
> {quote}
> It seems that this behaviour is caused by the 
> {{org.apache.jackrabbit.oak.jcr.Parallelized}} runner, used in the 
> {{AbstractRepositoryTest}}. JUnit submits a {{Runnable}} wrapping all the 
> test cases from a single class to the {{ThreadPoolScheduler#schedule()}} 
> method and then invokes {{#finished()}}. The latter waits for 10 minutes 
> until the submitted {{Runnable}} is done. However, in case of the 
> {{RepositoryTest}}, all tests takes more than 10 minutes. The {{finished()}} 
> method simply returns, JUnit invokes another test class, while the 
> {{RepositoryTest}} is still running in the background.
> In the attached patch, the {{finished()}} method waits indefinitely until all 
> tests are done. It may seem a bit radical, but after all it's the same 
> behaviour as in single-thread JUnit runner, which doesn't have internal 
> timeout as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3773) Include segment information in Segment.toString

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3773:
---
Labels: candidate_oak_1_0 candidate_oak_1_2  (was: )

> Include segment information in Segment.toString
> ---
>
> Key: OAK-3773
> URL: https://issues.apache.org/jira/browse/OAK-3773
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.13
>
>
> The segment information ({{Segment#getSegmentInfo}}) should also be in the 
> dump of a segment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052730#comment-15052730
 ] 

Julian Reschke commented on OAK-3743:
-

"withAsyncIndexing" at least could help explaining why it only happens 
sometimes...

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OAK-3765) Parallelized test runner does not wait for test completion

[Julian Reschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke resolved OAK-3765.
-
Resolution: Fixed

trunk: http://svn.apache.org/r1719288
1.2: http://svn.apache.org/r1719414
1.0: http://svn.apache.org/r1719422


> Parallelized test runner does not wait for test completion
> --
>
> Key: OAK-3765
> URL: https://issues.apache.org/jira/browse/OAK-3765
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: jcr
>Affects Versions: 1.2.9, 1.0.25, 1.3.12
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Fix For: 1.3.13, 1.0.26, 1.2.10
>
>
> As analyzed by [~tomek.rekawek] in OAK-3743:
> Quote from the mailing list:
> {quote}
> On a “slow” machine, in the surefire logs for the AtomicCounterTest (which 
> takes 63 sec while it should 3 sec), following test case appears:
> {noformat}
>classname="org.apache.jackrabbit.oak.jcr.RepositoryTest" 
> name="importWithRegisteredType[RDBDocumentStore on 
> jdbc:derby:oaktest\;create=true]"/>
> {noformat}
> It’s a test case from a completely different class. I downloaded all the 
> surefire reports and it seems that the test cases from the RepositoryTest 
> class are spread across CRUDTest, ConflictResolutionTest, ObservationTest and 
> others. On the “fast” machines the problem doesn’t exist and all 
> RepositoryTest methods are invoked within their own test case.
> {quote}
> It seems that this behaviour is caused by the 
> {{org.apache.jackrabbit.oak.jcr.Parallelized}} runner, used in the 
> {{AbstractRepositoryTest}}. JUnit submits a {{Runnable}} wrapping all the 
> test cases from a single class to the {{ThreadPoolScheduler#schedule()}} 
> method and then invokes {{#finished()}}. The latter waits for 10 minutes 
> until the submitted {{Runnable}} is done. However, in case of the 
> {{RepositoryTest}}, all tests takes more than 10 minutes. The {{finished()}} 
> method simply returns, JUnit invokes another test class, while the 
> {{RepositoryTest}} is still running in the background.
> In the attached patch, the {{finished()}} method waits indefinitely until all 
> tests are done. It may seem a bit radical, but after all it's the same 
> behaviour as in single-thread JUnit runner, which doesn't have internal 
> timeout as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3342) move benchmarks in oak-development module

[Davide Giannella (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-3342:
--
Summary: move benchmarks in oak-development module  (was: oak-benchmarks 
module)

> move benchmarks in oak-development module
> -
>
> Key: OAK-3342
> URL: https://issues.apache.org/jira/browse/OAK-3342
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
>Priority: Minor
> Fix For: 1.4
>
>
> Investigate feasibility and split the benchmarks off the oak-run into
> own module not deployed during release.
> Should reduce size of oak-run and limit the changes to the latter.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[Thomas Mueller (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052725#comment-15052725
 ] 

Thomas Mueller commented on OAK-3743:
-

It would be good to have a few full thread dumps to know what's going on. What 
I have done in the past for similar cases is log a full thread dump in a 
background thread every 10 minutes. Basically call 
java.lang.Thread.getAllStackTraces() every 10 minutes and log the output.

I don't think "setFullTextComparisonWithoutIndex" has an effect on performance. 
More likely, "withAsyncIndexing" has an effect, but I also doubt it. We could 
disable both and see what happens (well sure, some tests would fail, but I 
guess that's not a problem if it's just done temporarily).



> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052724#comment-15052724
 ] 

Tomek Rękawek commented on OAK-3743:


Tests extending the AbstractRepositoryTest uses Parallelized runner. For each 
configured nsfixture there's a Runnable created and submitted to the executor 
service. The Runnable invokes all tests from the class, sequentially. After 
submitting the Runnable, the finished() method waited for 10 minutes until all 
tasks are done.

In our case we have just one nsfixture (=one Runnable submitted), but it's 
still ran in a separate thread.

If we have had many nsfixtures, they'd be run at the same time.

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3769) Issue with jcr:contains on tags property

[Rajeev Duggal (JIRA)]

Rajeev Duggal created OAK-3769:
--

 Summary: Issue with jcr:contains on tags property
 Key: OAK-3769
 URL: https://issues.apache.org/jira/browse/OAK-3769
 Project: Jackrabbit Oak
  Issue Type: Bug
  Components: lucene
Affects Versions: 1.2.8, 1.2.7
Reporter: Rajeev Duggal


Running the below query, results in Exception pointed by [1]

/jcr:root/content/dam//element(*,dam:Asset)[jcr:contains(jcr:content/metadata/@cq:tags,
 'stockphotography:business/business_abstract')] order by @jcr:created 
descending

Also if you remove the node at 
/oak:index/damAssetLucene/indexRules/dam:Asset/properties/cqTags  and re-index 
the /oak:index/damAssetLucene index, the query works.

Seems '/' is special character and needs to be escaped by Oak.

[1]
Caused by: org.apache.lucene.queryparser.flexible.core.QueryNodeParseException: 
Syntax Error, cannot parse stockphotography\:business/business_abstract: 
Lexical error at line 1, column 45.  Encountered:  after : 
"/business_abstract" 
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.parse(StandardSyntaxParser.java:74)
at 
org.apache.lucene.queryparser.flexible.core.QueryParserHelper.parse(QueryParserHelper.java:250)
at 
org.apache.lucene.queryparser.flexible.standard.StandardQueryParser.parse(StandardQueryParser.java:168)
at 
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.tokenToQuery(LucenePropertyIndex.java:1260)
... 138 common frames omitted
Caused by: 
org.apache.lucene.queryparser.flexible.standard.parser.TokenMgrError: Lexical 
error at line 1, column 45.  Encountered:  after : "/business_abstract"
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParserTokenManager.getNextToken(StandardSyntaxParserTokenManager.java:937)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.jj_scan_token(StandardSyntaxParser.java:945)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.jj_3R_4(StandardSyntaxParser.java:827)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.jj_3_2(StandardSyntaxParser.java:739)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.jj_2_2(StandardSyntaxParser.java:730)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.Clause(StandardSyntaxParser.java:318)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.ModClause(StandardSyntaxParser.java:303)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.ConjQuery(StandardSyntaxParser.java:234)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.DisjQuery(StandardSyntaxParser.java:204)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.Query(StandardSyntaxParser.java:166)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.TopLevelQuery(StandardSyntaxParser.java:147)
at 
org.apache.lucene.queryparser.flexible.standard.parser.StandardSyntaxParser.parse(StandardSyntaxParser.java:65)
... 141 common frames omitted



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3765) Parallelized test runner does not wait for test completion

[Julian Reschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-3765:

Fix Version/s: 1.0.26

> Parallelized test runner does not wait for test completion
> --
>
> Key: OAK-3765
> URL: https://issues.apache.org/jira/browse/OAK-3765
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: jcr
>Affects Versions: 1.2.9, 1.0.25, 1.3.12
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Fix For: 1.3.13, 1.0.26, 1.2.10
>
>
> As analyzed by [~tomek.rekawek] in OAK-3743:
> Quote from the mailing list:
> {quote}
> On a “slow” machine, in the surefire logs for the AtomicCounterTest (which 
> takes 63 sec while it should 3 sec), following test case appears:
> {noformat}
>classname="org.apache.jackrabbit.oak.jcr.RepositoryTest" 
> name="importWithRegisteredType[RDBDocumentStore on 
> jdbc:derby:oaktest\;create=true]"/>
> {noformat}
> It’s a test case from a completely different class. I downloaded all the 
> surefire reports and it seems that the test cases from the RepositoryTest 
> class are spread across CRUDTest, ConflictResolutionTest, ObservationTest and 
> others. On the “fast” machines the problem doesn’t exist and all 
> RepositoryTest methods are invoked within their own test case.
> {quote}
> It seems that this behaviour is caused by the 
> {{org.apache.jackrabbit.oak.jcr.Parallelized}} runner, used in the 
> {{AbstractRepositoryTest}}. JUnit submits a {{Runnable}} wrapping all the 
> test cases from a single class to the {{ThreadPoolScheduler#schedule()}} 
> method and then invokes {{#finished()}}. The latter waits for 10 minutes 
> until the submitted {{Runnable}} is done. However, in case of the 
> {{RepositoryTest}}, all tests takes more than 10 minutes. The {{finished()}} 
> method simply returns, JUnit invokes another test class, while the 
> {{RepositoryTest}} is still running in the background.
> In the attached patch, the {{finished()}} method waits indefinitely until all 
> tests are done. It may seem a bit radical, but after all it's the same 
> behaviour as in single-thread JUnit runner, which doesn't have internal 
> timeout as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3342) move benchmarks in oak-development module

[Davide Giannella (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-3342:
--
Description: Take all the benchmarks provided by oak-run and move them into 
the oak-development module. Micro-benchmarking and Scalability  (was: 
Investigate feasibility and split the benchmarks off the oak-run into
own module not deployed during release.

Should reduce size of oak-run and limit the changes to the latter.
)

> move benchmarks in oak-development module
> -
>
> Key: OAK-3342
> URL: https://issues.apache.org/jira/browse/OAK-3342
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
>Priority: Minor
> Fix For: 1.4
>
>
> Take all the benchmarks provided by oak-run and move them into the 
> oak-development module. Micro-benchmarking and Scalability



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-3743:

Comment: was deleted

(was: Comparing times for JCR QueryTest between the two latest builds (615-> 
success, 614-> aborted):

- in the "slow" build, it took over 6 minutes, and each individual test took at 
least 13s (see 
)

- in the "fast" build, it took  seconds (see 
)

The fact that the slowness affects all tests hints that it has to do with setup 
or teardown...)

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052701#comment-15052701
 ] 

Tomek Rękawek commented on OAK-3743:


Going further, all "slow" tests are children of the {{AbstractRepositoryTest}}, 
which takes the {{DocumentNodeStore}} and creates a JCR repository. I suppose 
that the {{Jcr()}} introduces the performance problems.

Another fact - the {{ReferenceBinaryIT}} which also creates a JCR repository, 
remains fast. The only difference in the JCR instantiation between 
{{AbstractRepositoryTest}} and {{ReferenceBinaryIT}} is that in the former 
there are two extra settings added:

{code}
QueryEngineSettings qs = new QueryEngineSettings();
qs.setFullTextComparisonWithoutIndex(true);
return jcr.withAsyncIndexing().with(qs);
{code}

So, I suppose that the cause of the problem lies here.

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3765) Parallelized test runner does not wait for test completion

[Julian Reschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-3765:

Fix Version/s: 1.2.10
   1.3.13

> Parallelized test runner does not wait for test completion
> --
>
> Key: OAK-3765
> URL: https://issues.apache.org/jira/browse/OAK-3765
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: jcr
>Affects Versions: 1.2.9, 1.0.25, 1.3.12
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Fix For: 1.3.13, 1.2.10
>
>
> As analyzed by [~tomek.rekawek] in OAK-3743:
> Quote from the mailing list:
> {quote}
> On a “slow” machine, in the surefire logs for the AtomicCounterTest (which 
> takes 63 sec while it should 3 sec), following test case appears:
> {noformat}
>classname="org.apache.jackrabbit.oak.jcr.RepositoryTest" 
> name="importWithRegisteredType[RDBDocumentStore on 
> jdbc:derby:oaktest\;create=true]"/>
> {noformat}
> It’s a test case from a completely different class. I downloaded all the 
> surefire reports and it seems that the test cases from the RepositoryTest 
> class are spread across CRUDTest, ConflictResolutionTest, ObservationTest and 
> others. On the “fast” machines the problem doesn’t exist and all 
> RepositoryTest methods are invoked within their own test case.
> {quote}
> It seems that this behaviour is caused by the 
> {{org.apache.jackrabbit.oak.jcr.Parallelized}} runner, used in the 
> {{AbstractRepositoryTest}}. JUnit submits a {{Runnable}} wrapping all the 
> test cases from a single class to the {{ThreadPoolScheduler#schedule()}} 
> method and then invokes {{#finished()}}. The latter waits for 10 minutes 
> until the submitted {{Runnable}} is done. However, in case of the 
> {{RepositoryTest}}, all tests takes more than 10 minutes. The {{finished()}} 
> method simply returns, JUnit invokes another test class, while the 
> {{RepositoryTest}} is still running in the background.
> In the attached patch, the {{finished()}} method waits indefinitely until all 
> tests are done. It may seem a bit radical, but after all it's the same 
> behaviour as in single-thread JUnit runner, which doesn't have internal 
> timeout as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052793#comment-15052793
 ] 

Julian Reschke commented on OAK-3743:
-

Comparing times for JCR QueryTest between the two latest builds (615-> success, 
614-> aborted):

- in the "slow" build, it took over 6 minutes, and each individual test took at 
least 13s (see 
)

- in the "fast" build, it took  seconds (see 
)

The fact that the slowness affects all tests hints that it has to do with setup 
or teardown...

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3770) Move the Groovy console into oak-operations module

[Davide Giannella (JIRA)]

Davide Giannella created OAK-3770:
-

 Summary: Move the Groovy console into oak-operations module
 Key: OAK-3770
 URL: https://issues.apache.org/jira/browse/OAK-3770
 Project: Jackrabbit Oak
  Issue Type: Improvement
  Components: run
Affects Versions: 1.3.12
Reporter: Davide Giannella


Move the Groovy console from oak-run to oak-operations module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3771) Move backup into oak-operations

[Davide Giannella (JIRA)]

Davide Giannella created OAK-3771:
-

 Summary: Move backup into oak-operations
 Key: OAK-3771
 URL: https://issues.apache.org/jira/browse/OAK-3771
 Project: Jackrabbit Oak
  Issue Type: Improvement
  Components: run
Reporter: Davide Giannella
 Fix For: 1.3.12


Move the backup functionality from oak-run to oak-operations module.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OAK-3773) Include segment information in Segment.toString

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig resolved OAK-3773.

   Resolution: Fixed
Fix Version/s: 1.3.13

At http://svn.apache.org/viewvc?rev=1719434=rev the segment info is 
included in the dump of a segment if available. 

> Include segment information in Segment.toString
> ---
>
> Key: OAK-3773
> URL: https://issues.apache.org/jira/browse/OAK-3773
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.3.13
>
>
> The segment information ({{Segment#getSegmentInfo}}) should also be in the 
> dump of a segment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-2847) Dependency cleanup

[Davide Giannella (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-2847:
--
Issue Type: Epic  (was: Improvement)

> Dependency cleanup 
> ---
>
> Key: OAK-2847
> URL: https://issues.apache.org/jira/browse/OAK-2847
> Project: Jackrabbit Oak
>  Issue Type: Epic
>Reporter: Michael Dürig
>Assignee: Vikas Saurabh
>  Labels: technical_debt
> Fix For: 1.4
>
>
> Early in the next release cycle we should go through the list of Oak's 
> dependencies and decide whether we have candidates we want to upgrade and 
> remove orphaned dependencies. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052831#comment-15052831
 ] 

Julian Reschke commented on OAK-3743:
-

[ Comparing times for JCR QueryTest between the two latest builds 
(615-> success, 614-> aborted):

- in the "slow" build, it took over 6 minutes, and each individual test took at 
least 13s (see 
 Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3773) Include segment information in Segment.toString

[JIRA]

Michael Dürig created OAK-3773:
--

 Summary: Include segment information in Segment.toString
 Key: OAK-3773
 URL: https://issues.apache.org/jira/browse/OAK-3773
 Project: Jackrabbit Oak
  Issue Type: Improvement
  Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig


The segment information ({{Segment#getSegmentInfo}}) should also be in the dump 
of a segment. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052714#comment-15052714
 ] 

Michael Dürig commented on OAK-3743:


This was introduced here: 
http://svn.apache.org/viewvc?view=revision=1625638

Not sure why, and what effect this would have on the tests in general. Maybe 
[~tmueller] knows...

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052721#comment-15052721
 ] 

Julian Reschke commented on OAK-3743:
-

So how come that we came aware of the problem in the first place? If we don't 
run tests in parallel, why does the change have an effect?

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3766) Investigate and remove dependencies from oak-run

[Davide Giannella (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-3766:
--
Affects Version/s: 1.3.12
  Component/s: run

> Investigate and remove dependencies from oak-run
> 
>
> Key: OAK-3766
> URL: https://issues.apache.org/jira/browse/OAK-3766
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: run
>Affects Versions: 1.3.12
>Reporter: Davide Giannella
>
> While with OAK-3134 the effort is to split oak-run into smaller bundle to 
> ease deployment and maintenance this is to see if we can easily cut the size 
> of it by removing unused dependencies.
> *dependencies identified so far to investigate*
> - hadoop
> - solr
> - tika
> - jetty
> - zookeeper
> - H2 (SQL part)
> - JR remoting



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3772) Move restore into oak-operations

[Davide Giannella (JIRA)]

Davide Giannella created OAK-3772:
-

 Summary: Move restore into oak-operations
 Key: OAK-3772
 URL: https://issues.apache.org/jira/browse/OAK-3772
 Project: Jackrabbit Oak
  Issue Type: Improvement
  Components: run
Reporter: Davide Giannella
 Fix For: 1.3.12


Move the Restore functionality from oak-run into oak-operations



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052848#comment-15052848
 ] 

Tomek Rękawek commented on OAK-3743:


+1 to disabling withAsyncIndexing() for the weekend. [~tmueller], could you do 
it?

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OAK-3743) Build time out after 90 mins on Jenkins

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052724#comment-15052724
 ] 

Tomek Rękawek edited comment on OAK-3743 at 12/11/15 1:15 PM:
--

Tests extending the AbstractRepositoryTest uses Parallelized runner. For each 
configured nsfixture there's a Runnable created and submitted to the executor 
service. The Runnable invokes all tests from the class, sequentially. After 
submitting the Runnable, the finished() method waited for 10 minutes until all 
tasks are done.

In our case we have just one nsfixture (=one Runnable submitted), but it's 
still run in a separate thread.

If we have had many nsfixtures, they'd be run at the same time.


was (Author: tomek.rekawek):
Tests extending the AbstractRepositoryTest uses Parallelized runner. For each 
configured nsfixture there's a Runnable created and submitted to the executor 
service. The Runnable invokes all tests from the class, sequentially. After 
submitting the Runnable, the finished() method waited for 10 minutes until all 
tasks are done.

In our case we have just one nsfixture (=one Runnable submitted), but it's 
still ran in a separate thread.

If we have had many nsfixtures, they'd be run at the same time.

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3774) Tool for detecting references to pre compacted segments

[JIRA]

Michael Dürig created OAK-3774:
--

 Summary: Tool for detecting references to pre compacted segments
 Key: OAK-3774
 URL: https://issues.apache.org/jira/browse/OAK-3774
 Project: Jackrabbit Oak
  Issue Type: Technical task
  Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig


While OAK-3560 allows us to detect reference to pre compacted segments through 
manual inspection, we also need tooling to help detect such cases on site, 
during longevity tests and for UT/IT.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3774) Tool for detecting references to pre compacted segments

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Dürig updated OAK-3774:
---
Labels: compaction gc tooling  (was: compaction gc)

> Tool for detecting references to pre compacted segments
> ---
>
> Key: OAK-3774
> URL: https://issues.apache.org/jira/browse/OAK-3774
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>  Labels: compaction, gc, tooling
> Fix For: 1.4
>
>
> While OAK-3560 allows us to detect reference to pre compacted segments 
> through manual inspection, we also need tooling to help detect such cases on 
> site, during longevity tests and for UT/IT.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-2472) Add support for atomic counters on cluster solutions

[Davide Giannella (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052925#comment-15052925
 ] 

Davide Giannella commented on OAK-2472:
---

The main implementation is now complete and this is actually waiting for 
OAK-3529 to finish the last bits and see if the route taken will actually work

> Add support for atomic counters on cluster solutions
> 
>
> Key: OAK-2472
> URL: https://issues.apache.org/jira/browse/OAK-2472
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 1.3.0
>Reporter: Davide Giannella
>Assignee: Davide Giannella
>  Labels: scalability
> Fix For: 1.4
>
> Attachments: atomic-counter.md
>
>
> As of OAK-2220 we added support for atomic counters in a non-clustered 
> situation. 
> This ticket is about covering the clustered ones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[Thomas Mueller (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052858#comment-15052858
 ] 

Thomas Mueller commented on OAK-3743:
-

http://svn.apache.org/r1719438

I don't think this will help anything, but let's see. On Tuesday, I can add a 
"Full Thread Dump" thread that logs every 5 minutes. That way, we can find out 
what the problem is without having to guess.

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (OAK-3743) Build time out after 90 mins on Jenkins

[Thomas Mueller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Mueller reassigned OAK-3743:
---

Assignee: Thomas Mueller

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>Assignee: Thomas Mueller
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3529) NodeStore API should expose an Instance ID

[Davide Giannella (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella updated OAK-3529:
--
Attachment: OAK-3529-6.patch

[~mduerig]

bq. Re. the patch, there is a problem in the order of initialisation: the store 
member is used before it is initialised in Jcr(Oak)

True. In [OAK-3529-6.patch|^OAK-3529-6.patch] there's a last attempt to 
workaround the fact that we expect the NodeStore to provide Clusterable. It's 
quite a refactoring. IMO it's better to go straight for adding 
{{.with(Clusterable}}.

Thoughts?

/cc [~mreutegg]

> NodeStore API should expose an Instance ID
> --
>
> Key: OAK-3529
> URL: https://issues.apache.org/jira/browse/OAK-3529
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: core
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.4
>
> Attachments: OAK-3529-1.patch, OAK-3529-2.patch, OAK-3529-3.patch, 
> OAK-3529-4.patch, OAK-3529-5.patch, OAK-3529-6.patch
>
>
> For better leveraging cluster oriented algorithm: discovery, atomic
> operations; it would be very helpful if the NodeStore could expose a
> unique instance id.
> This can be the same as a cluster ID.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052393#comment-15052393
 ] 

Julian Reschke edited comment on OAK-3743 at 12/11/15 8:29 AM:
---

Thumbs up!

As this likely won't fix *this* issue completely, I'll open a separate ticket, 
link that, test the patch, and apply it.


was (Author: reschke):
Thumbs up!

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3765) Parallelized test runner does not wait for test completion

[Julian Reschke (JIRA)]

Julian Reschke created OAK-3765:
---

 Summary: Parallelized test runner does not wait for test completion
 Key: OAK-3765
 URL: https://issues.apache.org/jira/browse/OAK-3765
 Project: Jackrabbit Oak
  Issue Type: Bug
  Components: jcr
Affects Versions: 1.3.12, 1.0.25, 1.2.9
Reporter: Julian Reschke
Assignee: Julian Reschke
Priority: Minor


As analyzed by [~tomek.rekawek] in OAK-3743:

Quote from the mailing list:
{quote}
On a “slow” machine, in the surefire logs for the AtomicCounterTest (which 
takes 63 sec while it should 3 sec), following test case appears:
{noformat}
  
{noformat}
It’s a test case from a completely different class. I downloaded all the 
surefire reports and it seems that the test cases from the RepositoryTest class 
are spread across CRUDTest, ConflictResolutionTest, ObservationTest and others. 
On the “fast” machines the problem doesn’t exist and all RepositoryTest methods 
are invoked within their own test case.
{quote}

It seems that this behaviour is caused by the 
{{org.apache.jackrabbit.oak.jcr.Parallelized}} runner, used in the 
{{AbstractRepositoryTest}}. JUnit submits a {{Runnable}} wrapping all the test 
cases from a single class to the {{ThreadPoolScheduler#schedule()}} method and 
then invokes {{#finished()}}. The latter waits for 10 minutes until the 
submitted {{Runnable}} is done. However, in case of the {{RepositoryTest}}, all 
tests takes more than 10 minutes. The {{finished()}} method simply returns, 
JUnit invokes another test class, while the {{RepositoryTest}} is still running 
in the background.

In the attached patch, the {{finished()}} method waits indefinitely until all 
tests are done. It may seem a bit radical, but after all it's the same 
behaviour as in single-thread JUnit runner, which doesn't have internal timeout 
as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (OAK-3734) Release Oak 1.3.12

[Davide Giannella (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella resolved OAK-3734.
---
Resolution: Fixed

> Release Oak 1.3.12
> --
>
> Key: OAK-3734
> URL: https://issues.apache.org/jira/browse/OAK-3734
> Project: Jackrabbit Oak
>  Issue Type: Task
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Fix For: 1.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (OAK-3748) Heuristic method to decide if the document may be a part of the bulk update

[JIRA]

 [ 
https://issues.apache.org/jira/browse/OAK-3748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomek Rękawek updated OAK-3748:
---
Summary: Heuristic method to decide if the document may be a part of the 
bulk update  (was: Heuristic to decide if the document may be a part of the 
bulk update)

> Heuristic method to decide if the document may be a part of the bulk update
> ---
>
> Key: OAK-3748
> URL: https://issues.apache.org/jira/browse/OAK-3748
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: mongomk, rdbmk
>Reporter: Tomek Rękawek
> Fix For: 1.4
>
> Attachments: OAK-3748.patch
>
>
> OAK-2066 introduces the bulk createOrUpdate method in the DocumentStore 
> interface and implements it in the Mongo and RDB DSes. Both implementations 
> try to apply the bulk changes in a few iterations. The iteration first reads 
> the current documents and then updates them. If there's a concurrent 
> modification between read and update, it causes a conflict. Such documents 
> will be updated in the next iteration. The documents that can't be updated in 
> a few trials are eventually updated in a sequential way.
> Some documents are more probable to cause a fail than other. Eg. the root 
> document is very likely to be changed by some other process. We could create 
> a list of such "hotspot" documents and exclude them from the bulk updates. 
> The list can be self-maintaining, eg. documents which conflicted in more than 
> 50% cases in the last 1h.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[Julian Reschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052393#comment-15052393
 ] 

Julian Reschke commented on OAK-3743:
-

Thumbs up!

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3743) Build time out after 90 mins on Jenkins

[JIRA]

[ 
https://issues.apache.org/jira/browse/OAK-3743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052693#comment-15052693
 ] 

Tomek Rękawek commented on OAK-3743:


Yes, the OAK-3765 fixed the logs issue.

The oak-jcr would only run tests in parallel if there have been multiple 
nsfixtures passed (which is not the case for the Jenkins job). Right now tests 
run in just one thread.

> Build time out after 90 mins on Jenkins
> ---
>
> Key: OAK-3743
> URL: https://issues.apache.org/jira/browse/OAK-3743
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: rdbmk
> Environment: Jenkins, Ubuntu: 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>  Labels: CI, Jenkins
> Fix For: 1.4
>
> Attachments: OAK-3743-parallelized.patch
>
>
> Every 2nd build or so is aborted after a time out (90 mins):
> {noformat}
> Build timed out (after 90 minutes). Marking the build as aborted.
> Build was aborted
> [FINDBUGS] Skipping publisher since build result is ABORTED
> Recording test results
> Finished: ABORTED
> {noformat}
> See  e.g. 
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/585/jdk=jdk1.8.0_11,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> Interestingly when successful, the build takes about 30mins, so way below the 
> 90 min. timeout. 
> Also seen at builds 587, 590, 591, 593, 595, 597, 598, 604, 608, 609, 610 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (OAK-3775) Inconsistency between Node.getPrimaryType and Node.isNodeType

[Amrit Verma (JIRA)]

Amrit Verma created OAK-3775:


 Summary:  Inconsistency between Node.getPrimaryType and 
Node.isNodeType
 Key: OAK-3775
 URL: https://issues.apache.org/jira/browse/OAK-3775
 Project: Jackrabbit Oak
  Issue Type: Bug
  Components: jcr
Reporter: Amrit Verma


Steps:

1. Create a node structure say /node1/node2 (node2 should be nt:hierarchyNode 
type node)
2. Create a testuser and grant read permission on whole repository.
3. Grant permission jcr:read to testuser on node2.
3. On node2, deny jcr:read to testuser with rep:glob=/*

Execute the following from testuser's session:
* {{node.getPrimaryNodeType().isNodeType(JcrConstants.NT_HIERARCHYNODE)}}
* {{node.isNodeType(JcrConstants.NT_HIERARCHYNODE)}}

where node refers to node2 created above. The first expression returns true but 
the second returns false. 
Expected: both should return true.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3758) Multiplexing store support in Reference index

[Chetan Mehrotra (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053162#comment-15053162
 ] 

Chetan Mehrotra commented on OAK-3758:
--

Initial support implemented in branch with 
[ab34c3d3d76|https://github.com/chetanmeh/jackrabbit-oak/commit/ab34c3d3d760af918174c2449cfee78ef1ca4675].

The resultant node structure looks like. Here 'foo' is the mount name with 
'oak:foo' being used as the name of mount in path. Reference index uses two 
hidden index tree one for references and other for weakreferences. So 
multiplexing logic would multiplex both the tree separately 

{noformat}
  /oak:index/reference{type = reference, jcr:primaryType = 
oak:QueryIndexDefinition}
:references
  u1
b
  foo{match = true}
:oak:foo-references
  u1
a
  x
foo{match = true}
:weakreferences
  u2
d
  foo{match = true}
  u1
c
  foo{match = true}
:oak:foo-weakreferences
  u1
a
  y
foo{match = true}
{noformat}

> Multiplexing store support in Reference index
> -
>
> Key: OAK-3758
> URL: https://issues.apache.org/jira/browse/OAK-3758
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OAK-3775) Inconsistency between Node.getPrimaryType and Node.isNodeType

[Amrit Verma (JIRA)]

[ 
https://issues.apache.org/jira/browse/OAK-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053178#comment-15053178
 ] 

Amrit Verma commented on OAK-3775:
--

[~anchela], Could you please have a look at this issue.

>  Inconsistency between Node.getPrimaryType and Node.isNodeType
> --
>
> Key: OAK-3775
> URL: https://issues.apache.org/jira/browse/OAK-3775
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: jcr
>Reporter: Amrit Verma
>
> Steps:
> 1. Create a node structure say /node1/node2 (node2 should be nt:hierarchyNode 
> type node)
> 2. Create a testuser and grant read permission on whole repository.
> 3. Grant permission jcr:read to testuser on node2.
> 3. On node2, deny jcr:read to testuser with rep:glob=/*
> Execute the following from testuser's session:
> * {{node.getPrimaryNodeType().isNodeType(JcrConstants.NT_HIERARCHYNODE)}}
> * {{node.isNodeType(JcrConstants.NT_HIERARCHYNODE)}}
> where node refers to node2 created above. The first expression returns true 
> but the second returns false. 
> Expected: both should return true.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (OAK-3686) Solr suggestion results should have 1 row per suggestion with appropriate column names

[Davide Giannella (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OAK-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Davide Giannella closed OAK-3686.
-

Bulk close for 1.3.12

> Solr suggestion results should have 1 row per suggestion with appropriate 
> column names
> --
>
> Key: OAK-3686
> URL: https://issues.apache.org/jira/browse/OAK-3686
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: solr
>Reporter: Vikas Saurabh
>Assignee: Vikas Saurabh
>Priority: Minor
> Fix For: 1.3.12
>
> Attachments: OAK-3686.patch
>
>
> Currently suggest query returns just one row with {{rep:suggest()}} column 
> containing a string that needs to be parsed.
> It'd better if each suggestion is returned as individual row with column 
> names such as {{suggestion}}, {{weight}}(???), etc.
> (This is essentially the same issue as OAK-3509 but for solr)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)