Chetan,
When running locally with similar data, the indexing is nearly
instantaneous. Here¹s the output from persisting data; the first line is
the actual creation of the asset:
2015-12-07T09:29:37,871-0600 INFO ServiceLogger:53 - Call to:
com.target.redoak.controller.page.PageController.createPage executed in:
1477ms
2015-12-07T09:29:39,205-0600 DEBUG AsyncIndexUpdate:266 - Running
background index task async
2015-12-07T09:29:39,266-0600 DEBUG AsyncIndexUpdate:216 - Releasing
temporary checkpoint r1517d0f198b-0-25: true
2015-12-07T09:29:39,534-0600 INFO LuceneIndexEditorContext:359 - Loaded
default Tika Config from classpath
jar:file:/Users/z013slw/Documents/mars/.metadata/.plugins/org.eclipse.wst.s
erver.core/tmp1/wtpwebapps/red-oak-cms/WEB-INF/lib/oak-lucene-1.2.8.jar!/or
g/apache/jackrabbit/oak/plugins/index/lucene/tika-config.xml
2015-12-07T09:29:39,617-0600 DEBUG perf:183 - Created IndexWriter for
directory Lucene Index :
ComponentPathIndex(/pages/oak:index/ComponentPathIndex) [took 52ms]
2015-12-07T09:29:39,648-0600 DEBUG perf:183 - Created IndexWriter for
directory Lucene Index : PageIndex(/pages/oak:index/PageIndex) [took 2ms]
2015-12-07T09:29:39,660-0600 DEBUG perf:183 - Completed suggester for
directory Lucene Index :
ComponentPathIndex(/pages/oak:index/ComponentPathIndex) [took 0ms]
2015-12-07T09:29:39,721-0600 DEBUG perf:183 - Closed writer for directory
Lucene Index : ComponentPathIndex(/pages/oak:index/ComponentPathIndex)
[took 61ms]
2015-12-07T09:29:39,722-0600 DEBUG perf:183 - Closed directory for
directory Lucene Index :
ComponentPathIndex(/pages/oak:index/ComponentPathIndex) [took 62ms]
2015-12-07T09:29:39,723-0600 DEBUG perf:183 - Overall Closed IndexWriter
for directory Lucene Index :
ComponentPathIndex(/pages/oak:index/ComponentPathIndex) [took 63ms]
2015-12-07T09:29:39,724-0600 DEBUG LuceneIndexEditorContext:400 - Text
extraction stats 0 (Time Taken 0 min, 0 sec, Bytes Read 0 B, Extracted
text size 0 B)
2015-12-07T09:29:39,724-0600 DEBUG LuceneIndexEditor:218 -
[ComponentPathIndex(/pages/oak:index/ComponentPathIndex)] => Indexed 4
nodes, done.
2015-12-07T09:29:39,724-0600 DEBUG perf:183 - Completed suggester for
directory Lucene Index : PageIndex(/pages/oak:index/PageIndex) [took 0ms]
2015-12-07T09:29:39,742-0600 DEBUG perf:183 - Closed writer for directory
Lucene Index : PageIndex(/pages/oak:index/PageIndex) [took 18ms]
2015-12-07T09:29:39,742-0600 DEBUG perf:183 - Closed directory for
directory Lucene Index : PageIndex(/pages/oak:index/PageIndex) [took 18ms]
2015-12-07T09:29:39,744-0600 DEBUG perf:183 - Overall Closed IndexWriter
for directory Lucene Index : PageIndex(/pages/oak:index/PageIndex) [took
20ms]
2015-12-07T09:29:39,744-0600 DEBUG LuceneIndexEditorContext:400 - Text
extraction stats 0 (Time Taken 0 min, 0 sec, Bytes Read 0 B, Extracted
text size 0 B)
2015-12-07T09:29:39,744-0600 DEBUG LuceneIndexEditor:218 -
[PageIndex(/pages/oak:index/PageIndex)] => Indexed 1 nodes, done.
2015-12-07T09:29:39,744-0600 DEBUG IndexUpdate:259 - Indexing report
- /pages/oak:index/ComponentPathIndex(4)
- /pages/oak:index/PageIndex(1)
2015-12-07T09:29:39,776-0600 DEBUG perf:183 -
[/pages/oak:index/ComponentPathIndex] Index found to be updated. Reopening
the IndexNode [took 8ms]
2015-12-07T09:29:39,784-0600 DEBUG perf:183 - [/pages/oak:index/PageIndex]
Index found to be updated. Reopening the IndexNode [took 7ms]
2015-12-07T09:29:39,795-0600 DEBUG AsyncIndexUpdate:424 - AsyncIndex
(async) update run completed in 529.3 ms. Indexed 5 nodes
So it appears there is some kind of contention when there are multiple
JVMs involved.
Jim
On 12/6/15, 11:45 PM, "Chetan Mehrotra" <[email protected]> wrote:
>Hi Jim,
>
>How does the indexing performs if you say just run a single webapp node?
>
>Chetan Mehrotra
>
>
>On Sat, Dec 5, 2015 at 7:18 AM, Jim.Tully <[email protected]> wrote:
>> We are using Oak embedded in a web application, and are now
>>experiencing significant delays in async indexing. New nodes added are
>>sometimes not available by query for up to an hour. I¹m hoping you can
>>identify areas I might explore to improve this performance.
>>
>> We have multiple instances of the web application running with the same
>>Mongodb cluster connected via SSL. Our Repository constructor is:
>>
>>
>>
>> ns = new
>>DocumentMK.Builder().setMongoDB(createMongoDB()).getNodeStore();
>>
>>
>> Oak oak = new Oak(ns);
>>
>>
>> LuceneIndexProvider provider = new LuceneIndexProvider();
>>
>> Jcr jcr = new Jcr(oak).with((QueryIndexProvider)
>>provider).with((Observer) provider)
>>
>> .with(new LuceneIndexEditorProvider()).withAsyncIndexing();
>>
>> repository = jcr.createRepository();
>>
>>
>> The web application creates the repository at start up, and disposes of
>>it as shutdown. We have no observers registered at all, but do have 6
>>lucene indexes defined. The index that is currently giving me heartburn
>>looks like below. Where would I start to find what is dragging
>>performance down so drastically?
>>
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>>
>> <sv:node xmlns:sv="http://www.jcp.org/jcr/sv/1.0"; sv:name="PageIndex">
>>
>> <sv:property sv:name="jcr:primaryType" sv:type="Name">
>>
>> <sv:value>oak:QueryIndexDefinition</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="compatVersion" sv:type="Long">
>>
>> <sv:value>2</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="compatMode" sv:type="Long">
>>
>> <sv:value>2</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="type" sv:type="String">
>>
>> <sv:value>lucene</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="async" sv:type="String">
>>
>> <sv:value>async</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="name" sv:type="String">
>>
>> <sv:value>PageIndex</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="indexPath" sv:type="String">
>>
>> <sv:value>/pages/oak:index/PageIndex</sv:value>
>>
>> </sv:property>
>>
>>
>>
>> <sv:node sv:name="indexRules" sv:type="Name">
>>
>> <sv:property sv:name="jcr:primaryType" sv:type="Name">
>>
>> <sv:value>nt:unstructured</sv:value>
>>
>> </sv:property>
>>
>> <sv:node sv:name="tgt:page">
>>
>> <sv:property sv:name="jcr:primaryType" sv:type="Name">
>>
>> <sv:value>nt:unstructured</sv:value>
>>
>> </sv:property>
>>
>> <sv:node sv:name="properties" sv:type="Name">
>>
>> <sv:property sv:name="jcr:primaryType" sv:type="Name">
>>
>> <sv:value>nt:unstructured</sv:value>
>>
>> </sv:property>
>>
>> <sv:node sv:name="jcr:activationDate">
>>
>> <sv:property sv:name="jcr:primaryType" sv:type="Name">
>>
>> <sv:value>nt:unstructured</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="type" sv:type="String">
>>
>> <sv:value>Date</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="ordered" sv:type="Boolean">
>>
>> <sv:value>true</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="propertyIndex" sv:type="Boolean">
>>
>> <sv:value>true</sv:value>
>>
>> </sv:property>
>>
>> </sv:node>
>>
>> <sv:node sv:name="jcr:deactivationDate">
>>
>> <sv:property sv:name="jcr:primaryType" sv:type="Name">
>>
>> <sv:value>nt:unstructured</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="type" sv:type="String">
>>
>> <sv:value>Date</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="ordered" sv:type="Boolean">
>>
>> <sv:value>true</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="propertyIndex" sv:type="Boolean">
>>
>> <sv:value>true</sv:value>
>>
>> </sv:property>
>>
>> </sv:node>
>>
>> <sv:node sv:name="jcr:status">
>>
>> <sv:property sv:name="jcr:primaryType" sv:type="Name">
>>
>> <sv:value>nt:unstructured</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="propertyIndex" sv:type="Boolean">
>>
>> <sv:value>true</sv:value>
>>
>> </sv:property>
>>
>> </sv:node>
>>
>> <sv:node sv:name="presentation">
>>
>> <sv:property sv:name="jcr:primaryType" sv:type="Name">
>>
>> <sv:value>nt:unstructured</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="propertyIndex" sv:type="Boolean">
>>
>> <sv:value>true</sv:value>
>>
>> </sv:property>
>>
>> </sv:node>
>>
>> </sv:node>
>>
>> </sv:node>
>>
>> </sv:node>
>>
>> <sv:node sv:name="analyzers">
>>
>> <sv:node sv:name="default">
>>
>> <sv:property sv:name="class" sv:type="String">
>>
>>
>><sv:value>org.apache.lucene.analysis.standard.StandardAnalyzer</sv:value>
>>
>> </sv:property>
>>
>> <sv:property sv:name="luceneMatchVersion" sv:type="String">
>>
>> <sv:value>LUCENE_47</sv:value>
>>
>> </sv:property>
>>
>> <sv:node sv:name="tokenizer">
>>
>> <sv:property sv:name="name" sv:type="String">
>>
>> <sv:value>Standard</sv:value>
>>
>> </sv:property>
>>
>> </sv:node>
>>
>> </sv:node>
>>
>> </sv:node>
>>
>> </sv:node>
>>
>> Thanks,
>>
>> Jim
>>
>
