Will do.  I need to sanitize the unit test a bit so it might be a few days.


On Fri, Jan 17, 2014 at 9:24 PM, Shai Erera <ser...@gmail.com> wrote:

> Can you open an issue and attach the test there?
> On Jan 18, 2014 12:41 AM, "Matthew Petersen" <mdpe...@gmail.com> wrote:
>
> > I do have a test that will reproduce.  I'm not adding categories with
> very
> > deep hierarchies, I'm adding 129 category paths per document (all docs
> have
> > paths with same label) with each path having one value.  All of the
> values
> > are completely random and likely unique.  It's basically a worst case
> test
> > for our app but the condition has been seen in the field (the error has
> > been encountered at less than worst case scenario).  The test i have
> > reproduces it very quickly, Only have to index ~330K docs.
> >
> >
> > On Fri, Jan 17, 2014 at 3:27 PM, Shai Erera <ser...@gmail.com> wrote:
> >
> > > Do you have a test which reproduces the error? Are you adding
> categories
> > > with very deep hierarchies?
> > >
> > > Shai
> > >
> > >
> > > On Fri, Jan 17, 2014 at 11:59 PM, Matthew Petersen <mdpe...@gmail.com
> > > >wrote:
> > >
> > > > I've confirmed that using the LruTaxonomyWriterCache solves the issue
> > for
> > > > me.  It would appear there is in fact a bug in the
> > > Cl20TaxonomyWriterCache
> > > > or I am using it incorrectly (I use it as default, no customization).
> > > >
> > > >
> > > > On Fri, Jan 17, 2014 at 9:29 AM, Matthew Petersen <mdpe...@gmail.com
> >
> > > > wrote:
> > > >
> > > > > I'm sure.  I had seen that issue and it looked similar but the
> stack
> > > > trace
> > > > > is slightly different.  I've found that if I replace the
> > > > > Cl2oTaxonomyWriterCache with the LruTaxonomyWriterCache the problem
> > > seems
> > > > > to go away.  I'm working right now on running a test that will
> prove
> > > this
> > > > > but it takes a while as the cache needs to get very large.  If this
> > > > proves
> > > > > to solve the problem then I'd say there is still a bug in the
> > > > > Cl2oTaxonomyWriterCache implementation.
> > > > >
> > > > > Thanks for the response.
> > > > > Matt
> > > > >
> > > > >
> > > > > On Fri, Jan 17, 2014 at 6:36 AM, Michael McCandless <
> > > > > luc...@mikemccandless.com> wrote:
> > > > >
> > > > >> Are you sure you're using 4.4?
> > > > >>
> > > > >> Because ... this looks like
> > > > >> https://issues.apache.org/jira/browse/LUCENE-5048 but that was
> > > > >> supposedly fixed in 4.4.
> > > > >>
> > > > >> Mike McCandless
> > > > >>
> > > > >> http://blog.mikemccandless.com
> > > > >>
> > > > >>
> > > > >> On Thu, Jan 16, 2014 at 5:33 PM, Matthew Petersen <
> > mdpe...@gmail.com>
> > > > >> wrote:
> > > > >> > I’m having an issue with an index when adding category paths to
> a
> > > > >> document.
> > > > >> >  They seem to be added without issue for a long period of time,
> > then
> > > > for
> > > > >> > some unknown reason the addition fails with an
> > ArrayIndexOutOfBounds
> > > > >> > exception.  Subsequent attempts to add category paths fail with
> > the
> > > > same
> > > > >> > exception.  I’ve run CheckIndex on both the index and the
> taxonomy
> > > > >> > directory and both come back as clean with no issues.  I cannot
> > fix
> > > > the
> > > > >> > index because according to lucene it is not broken.  Could this
> > be a
> > > > >> bug in
> > > > >> > lucene?  Below is the stack trace when the exception occurs:
> > > > >> >
> > > > >> >
> > > > >> > Lucene v4.4.0
> > > > >> >
> > > > >> >
> > > > >> > java.lang.ArrayIndexOutOfBoundsException: -65535
> > > > >> >
> > > > >> > at java.util.ArrayList.elementData(ArrayList.java:371)
> > > > >> >
> > > > >> > at java.util.ArrayList.get(ArrayList.java:384)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CharBlockArray.charAt(CharBlockArray.java:152)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CategoryPathUtils.equalsToSerialized(CategoryPathUtils.java:61)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:257)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.CompactLabelToOrdinal.getOrdinal(CompactLabelToOrdinal.java:140)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.writercache.cl2o.Cl2oTaxonomyWriterCache.get(Cl2oTaxonomyWriterCache.java:74)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyWriter.addCategory(DirectoryTaxonomyWriter.java:455)
> > > > >> >
> > > > >> > at
> > > > >>
> > > >
> > org.apache.lucene.facet.index.FacetFields.addFields(FacetFields.java:175)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.getDocument(LogIndexerImpl.java:478)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLog(LogIndexerImpl.java:392)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.logrhythm.messaging.indexing.LogIndexerImpl.indexLogs(LogIndexerImpl.java:357)
> > > > >> >
> > > > >> > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > >> >
> > > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.logrhythm.tests.unit.messaging.indexing.LogIndexerTests.logIndexerLoadTest(LogIndexerTests.java:752)
> > > > >> >
> > > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > >> >
> > > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> > > > >> >
> > > > >> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> > > > >> >
> > > > >> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > > >> >
> > > > >> > at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > > >> >
> > > > >> > at
> > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > > >> >
> > > > >> > at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > > >> >
> > > > >> > at
> > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > > >> >
> > > > >> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > > >> >
> > > > >> > at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:202)
> > > > >> >
> > > > >> > at
> > > > >>
> > > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> > > > >> >
> > > > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > > > >> >
> > > > >> > at
> > > > >> >
> > > > >>
> > > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > > >> >
> > > > >> > at java.lang.reflect.Method.invoke(Method.java:601)
> > > > >> >
> > > > >> > at
> > > > com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> > > > >>
> > > > >>
> > ---------------------------------------------------------------------
> > > > >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> > > > >> For additional commands, e-mail: java-user-h...@lucene.apache.org
> > > > >>
> > > > >>
> > > > >
> > > >
> > >
> >
>

Reply via email to