Re: Infinite loop

[Thiago Sanches]

Hi Chetan, thanks for your support.
I'll look for adobe suport.

On Fri, Sep 16, 2016 at 1:22 AM, Chetan Mehrotra 
wrote:

> Looks like index would need to be reindex. It would be better to
> contact Adobe Support as closer analysis would be required
> Chetan Mehrotra
>
>
> On Thu, Sep 15, 2016 at 6:32 PM, Thiago Sanches  wrote:
> > I removed the index folder but the error persists. I tried to remove the
> > "/crx/packmgr/service.jsp/file" node (that was causing the error before)
> > But still failing...
> >
> > 15.09.2016 12:58:31.417 *DEBUG* [pool-7-thread-3]
> > org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate [async] The
> index
> > update is still failing
> > org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0005:
> Failed
> > to remove the index entries of the removed subtree
> > /crx/packmgr/service.jsp/file
> >
> > I know that we removed and maybe caused some other issue, but I don't
> know
> > why the cause of the first error (before the node deletion).
> >
> > On Thu, Sep 15, 2016 at 8:51 AM, Thiago Sanches 
> wrote:
> >
> >> Hello Chetan, good morning.
> >>
> >> Yes, there is a lot of ".bak" inside segmentstore folder with the
> creation
> >> time close to the "force" restart.
> >> I'll try to remove the index folder.
> >>
> >> Thanks for your help.
> >>
> >> On Thu, Sep 15, 2016 at 1:44 AM, Chetan Mehrotra <
> >> chetan.mehro...@gmail.com> wrote:
> >>
> >>> On Thu, Sep 15, 2016 at 2:17 AM, Thiago Sanches 
> wrote:
> >>> > This issue start to appers after some problemas with disk space and
> some
> >>> > force restarts on AEM.
> >>>
> >>> Do you see presence of ".bak" files in segmentstore folder post system
> >>> restart after unclean shutdown with creation time close to subsequent
> >>> restart times? Can you try cleaning local index folder
> >>> (repository/index) and restart to see if its resolved. If not would
> >>> suggest to followup on Adobe Support portal
> >>>
> >>>
> >>> Chetan Mehrotra
> >>>
> >>
> >>
>

Re: Infinite loop

[Chetan Mehrotra]

Looks like index would need to be reindex. It would be better to
contact Adobe Support as closer analysis would be required
Chetan Mehrotra


On Thu, Sep 15, 2016 at 6:32 PM, Thiago Sanches  wrote:
> I removed the index folder but the error persists. I tried to remove the
> "/crx/packmgr/service.jsp/file" node (that was causing the error before)
> But still failing...
>
> 15.09.2016 12:58:31.417 *DEBUG* [pool-7-thread-3]
> org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate [async] The index
> update is still failing
> org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0005: Failed
> to remove the index entries of the removed subtree
> /crx/packmgr/service.jsp/file
>
> I know that we removed and maybe caused some other issue, but I don't know
> why the cause of the first error (before the node deletion).
>
> On Thu, Sep 15, 2016 at 8:51 AM, Thiago Sanches  wrote:
>
>> Hello Chetan, good morning.
>>
>> Yes, there is a lot of ".bak" inside segmentstore folder with the creation
>> time close to the "force" restart.
>> I'll try to remove the index folder.
>>
>> Thanks for your help.
>>
>> On Thu, Sep 15, 2016 at 1:44 AM, Chetan Mehrotra <
>> chetan.mehro...@gmail.com> wrote:
>>
>>> On Thu, Sep 15, 2016 at 2:17 AM, Thiago Sanches  wrote:
>>> > This issue start to appers after some problemas with disk space and some
>>> > force restarts on AEM.
>>>
>>> Do you see presence of ".bak" files in segmentstore folder post system
>>> restart after unclean shutdown with creation time close to subsequent
>>> restart times? Can you try cleaning local index folder
>>> (repository/index) and restart to see if its resolved. If not would
>>> suggest to followup on Adobe Support portal
>>>
>>>
>>> Chetan Mehrotra
>>>
>>
>>

Re: Infinite loop

[Thiago Sanches]

I removed the index folder but the error persists. I tried to remove the
"/crx/packmgr/service.jsp/file" node (that was causing the error before)
But still failing...

15.09.2016 12:58:31.417 *DEBUG* [pool-7-thread-3]
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate [async] The index
update is still failing
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0005: Failed
to remove the index entries of the removed subtree
/crx/packmgr/service.jsp/file

I know that we removed and maybe caused some other issue, but I don't know
why the cause of the first error (before the node deletion).

On Thu, Sep 15, 2016 at 8:51 AM, Thiago Sanches  wrote:

> Hello Chetan, good morning.
>
> Yes, there is a lot of ".bak" inside segmentstore folder with the creation
> time close to the "force" restart.
> I'll try to remove the index folder.
>
> Thanks for your help.
>
> On Thu, Sep 15, 2016 at 1:44 AM, Chetan Mehrotra <
> chetan.mehro...@gmail.com> wrote:
>
>> On Thu, Sep 15, 2016 at 2:17 AM, Thiago Sanches  wrote:
>> > This issue start to appers after some problemas with disk space and some
>> > force restarts on AEM.
>>
>> Do you see presence of ".bak" files in segmentstore folder post system
>> restart after unclean shutdown with creation time close to subsequent
>> restart times? Can you try cleaning local index folder
>> (repository/index) and restart to see if its resolved. If not would
>> suggest to followup on Adobe Support portal
>>
>>
>> Chetan Mehrotra
>>
>
>

Re: Infinite loop

[Thiago Sanches]

Hello Chetan, good morning.

Yes, there is a lot of ".bak" inside segmentstore folder with the creation
time close to the "force" restart.
I'll try to remove the index folder.

Thanks for your help.

On Thu, Sep 15, 2016 at 1:44 AM, Chetan Mehrotra 
wrote:

> On Thu, Sep 15, 2016 at 2:17 AM, Thiago Sanches  wrote:
> > This issue start to appers after some problemas with disk space and some
> > force restarts on AEM.
>
> Do you see presence of ".bak" files in segmentstore folder post system
> restart after unclean shutdown with creation time close to subsequent
> restart times? Can you try cleaning local index folder
> (repository/index) and restart to see if its resolved. If not would
> suggest to followup on Adobe Support portal
>
>
> Chetan Mehrotra
>

Re: Infinite loop

[Thiago Sanches]

Hello Vikas, good morning.



*Just to confirm: does the drive which contains  folderhave
sufficient space now? Any chance that AEM process can't write
to/repository/index/* folders?*

Yes, because the related problem with the repository growing too fast.


On Wed, Sep 14, 2016 at 6:03 PM, Vikas Saurabh 
wrote:

> Hi Thiago,
>
> > This issue start to appers after some problemas with disk space and some
> > force restarts on AEM.
>
> Just to confirm: does the drive which contains  folder
> have sufficient space now? Any chance that AEM process can't write to
> /repository/index/* folders?
>
> > The Oak Core version is: org.apache.jackrabbit.oak-core 1.2.11
>
> Good to know... but, I'm afraid my knowledge is almost already
> exhausted. Someone else on the list might come up with better ideas
> :).
>
> Thanks,
> Vikas
>

Re: Infinite loop

[Chetan Mehrotra]

On Thu, Sep 15, 2016 at 2:17 AM, Thiago Sanches  wrote:
> This issue start to appers after some problemas with disk space and some
> force restarts on AEM.

Do you see presence of ".bak" files in segmentstore folder post system
restart after unclean shutdown with creation time close to subsequent
restart times? Can you try cleaning local index folder
(repository/index) and restart to see if its resolved. If not would
suggest to followup on Adobe Support portal


Chetan Mehrotra

Re: Infinite loop

[Vikas Saurabh]

Hi Thiago,

> This issue start to appers after some problemas with disk space and some
> force restarts on AEM.

Just to confirm: does the drive which contains  folder
have sufficient space now? Any chance that AEM process can't write to
/repository/index/* folders?

> The Oak Core version is: org.apache.jackrabbit.oak-core 1.2.11

Good to know... but, I'm afraid my knowledge is almost already
exhausted. Someone else on the list might come up with better ideas
:).

Thanks,
Vikas

Re: Infinite loop

[Vikas Saurabh]

Hi Thiago,

> By property indices did you mean the 'propertyIndex' attribute?
No, I had meant value of "type" and "async" property on the index
definition - in your case it's "lucene" and "async" respectively.

> Caused by: java.io.FileNotFoundException: segments_3lxu
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory.openInput(IndexCopier.java:718)
> at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:324)

That would usually indicate some discrepancy between file content
stored in the repository and on filesystem (ignore the details if
you're new to this as you said earlier).

Would you remember what action caused this issue to appear? In the
past, we've seen cases where index definitions were installed using
AEM's package manager leading to such issues. Another reason has been
using ACS Common's Ensure oak index tool (btw, that issue has been
since resolved in ensure oak index).

BTW, which oak version are you using? (in your AEM installation, you
can go to /system/console and find version for oak-core)

Thanks,
Vikas

Re: Infinite loop

[Thiago Sanches]

Just to let you know.
We enabled the debug for async index and here are the result:

14.09.2016 19:57:05.073 *DEBUG* [pool-12-thread-5]
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate Running background
index task async
14.09.2016 19:57:05.074 *DEBUG* [pool-12-thread-5]
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate Releasing
temporary checkpoint 1e62c624-e26f-4e7b-aa21-f4172e0d0237: true
14.09.2016 19:57:07.980 *DEBUG* [pool-12-thread-5]
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate [async] The index
update is still failing
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed
to index the node /crx/packmgr/service.jsp/file
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:306)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:198)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:153)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:531)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:531)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:531)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord$2.childNodeChanged(MapRecord.java:403)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:487)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:394)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:583)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.updateIndex(AsyncIndexUpdate.java:489)
at
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.runWhenPermitted(AsyncIndexUpdate.java:431)
at
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.run(AsyncIndexUpdate.java:324)
at
org.apache.sling.commons.scheduler.impl.QuartzJobExecutor.execute(QuartzJobExecutor.java:115)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: segments_3lxu
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory.openInput(IndexCopier.java:718)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:324)
at
org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:404)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:843)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:694)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:400)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:746)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext.getWriter(LuceneIndexEditorContext.java:172)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:302)
... 23 common frames omitted



On Wed, Sep 14, 2016 at 4:56 PM, Thiago Sanches <tsi...@gmail.com> wrote:

> Hello Vikas, thanks for your reply.
>
> By property indices did you mean the 'propertyIndex' attribute?
>
> For example, we are using the following index definition:
> 
> http://jackrabbit.apache.org/oak/ns/1.0;
>  xmlns:sling="http://sling.apache.org/jcr/sling/1.0;
>  xmlns:jcr="http://www.jcp.org/jcr/1.0;
>  xmlns:nt="http://www.jcp.org/jcr/nt/1.0;
>  xmlns:rep="internal"
>  jcr:primaryType="oak:Unstructured"
>  async="async"
>  compatVersion="{Long}2"
>  evaluatePathRestrictions="{Boolean}true"
>  includedPaths="[/home]"
>  queryPaths="[/home]"
>  reindex="{Boolean}false"
>  reindexCount="{Long}6"
>  type="lucene">
> 
> 
> 
>  jcr:primaryType="nt:unstructured"/>
> 
> 
> 
>

Re: Infinite loop

[Thiago Sanches]

Hello Vikas, thanks for your reply.

By property indices did you mean the 'propertyIndex' attribute?

For example, we are using the following index definition:

http://jackrabbit.apache.org/oak/ns/1.0;
 xmlns:sling="http://sling.apache.org/jcr/sling/1.0;
 xmlns:jcr="http://www.jcp.org/jcr/1.0;
 xmlns:nt="http://www.jcp.org/jcr/nt/1.0;
 xmlns:rep="internal"
 jcr:primaryType="oak:Unstructured"
 async="async"
 compatVersion="{Long}2"
 evaluatePathRestrictions="{Boolean}true"
 includedPaths="[/home]"
 queryPaths="[/home]"
 reindex="{Boolean}false"
 reindexCount="{Long}6"
 type="lucene">









And this guy are stucked in a infinite loop.
Could be related with some corruption on repository?

Thanks.


On Wed, Sep 14, 2016 at 4:23 PM, Vikas Saurabh <vikas.saur...@gmail.com>
wrote:

> Hi Thiago,
>
> That most often happens with async index updates. Logger name in this
> case for log message for the loop you're describing would have
> "AsyncIndexUpdate".
> You can enabled DEBUG log for
> "org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate" which
> should then log an exception the next time around it restarts
> (Actually, it would have logged some error the first time around it
> hit the exception).
>
> Otoh, if the index is covering property indices, then you might be
> hitting some other issue(s).
>
> Thanks,
> Vikas
>
> On Thu, Sep 15, 2016 at 12:29 AM, Thiago Sanches <tsi...@gmail.com> wrote:
> > Hello guys,
> >
> > I'm new with oak indexes and I'm using it with Adobe AEM. Here we created
> > some indexes that it's seems that they are stucked in a infinite looping
> > for example:
> >
> > Reindexing Traversed #4 ...
> > Reindexing Traversed #5 ...
> > Reindexing Traversed #6 ...
> >
> > This process are decreasing the system performance.
> > I could not find any information about that.
> > Could you have any advice?
> >
> > Thanks.
>

Re: Infinite loop

[Vikas Saurabh]

Hi Thiago,

That most often happens with async index updates. Logger name in this
case for log message for the loop you're describing would have
"AsyncIndexUpdate".
You can enabled DEBUG log for
"org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate" which
should then log an exception the next time around it restarts
(Actually, it would have logged some error the first time around it
hit the exception).

Otoh, if the index is covering property indices, then you might be
hitting some other issue(s).

Thanks,
Vikas

On Thu, Sep 15, 2016 at 12:29 AM, Thiago Sanches  wrote:
> Hello guys,
>
> I'm new with oak indexes and I'm using it with Adobe AEM. Here we created
> some indexes that it's seems that they are stucked in a infinite looping
> for example:
>
> Reindexing Traversed #4 ...
> Reindexing Traversed #5 ...
> Reindexing Traversed #6 ...
>
> This process are decreasing the system performance.
> I could not find any information about that.
> Could you have any advice?
>
> Thanks.

Infinite loop

[Thiago Sanches]

Hello guys,

I'm new with oak indexes and I'm using it with Adobe AEM. Here we created
some indexes that it's seems that they are stucked in a infinite looping
for example:

Reindexing Traversed #4 ...
Reindexing Traversed #5 ...
Reindexing Traversed #6 ...

This process are decreasing the system performance.
I could not find any information about that.
Could you have any advice?

Thanks.