LuceneTextValueEncoder doesn't properly set internal buffers, causing
BufferUnderflowException
----------------------------------------------------------------------------------------------
Key: MAHOUT-855
URL: https://issues.apache.org/jira/browse/MAHOUT-855
Project: Mahout
Issue Type: Bug
Reporter: Grant Ingersoll
Assignee: Grant Ingersoll
Priority: Minor
Fix For: 0.6
The LuceneTextValueEncoder throws an BufferUnderflowException when used. See
the code below. The problem appears to be due to the CharBuffer not getting
values, but I'm not sure yet.
{code}
@Test
public void testLucene() throws Exception {
LuceneTextValueEncoder enc = new LuceneTextValueEncoder("text");
enc.setAnalyzer(new WhitespaceAnalyzer(Version.LUCENE_34));
Vector v1 = new DenseVector(200);
enc.addToVector("test1 and more", v1);
enc.flush(1, v1);
}
{code}
Here's the exception:
{quote}
java.nio.BufferUnderflowException
at java.nio.HeapCharBuffer.get(HeapCharBuffer.java:127)
at
org.apache.mahout.vectorizer.encoders.LuceneTextValueEncoder$CharSequenceReader.read(LuceneTextValueEncoder.java:87)
at org.apache.lucene.analysis.CharReader.read(CharReader.java:54)
at
org.apache.lucene.util.CharacterUtils$Java5CharacterUtils.fill(CharacterUtils.java:181)
at
org.apache.lucene.analysis.CharTokenizer.incrementToken(CharTokenizer.java:273)
at
org.apache.mahout.common.lucene.TokenStreamIterator.computeNext(TokenStreamIterator.java:41)
at
org.apache.mahout.common.lucene.TokenStreamIterator.computeNext(TokenStreamIterator.java:30)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
at
org.apache.mahout.vectorizer.encoders.TextValueEncoder.addText(TextValueEncoder.java:78)
at
org.apache.mahout.vectorizer.encoders.TextValueEncoder.addText(TextValueEncoder.java:69)
at
org.apache.mahout.vectorizer.encoders.TextValueEncoder.addToVector(TextValueEncoder.java:59)
at
org.apache.mahout.vectorizer.encoders.FeatureVectorEncoder.addToVector(FeatureVectorEncoder.java:86)
at
org.apache.mahout.vectorizer.encoders.FeatureVectorEncoder.addToVector(FeatureVectorEncoder.java:63)
at
org.apache.mahout.vectorizer.encoders.TextValueEncoderTest.testLucene(TextValueEncoderTest.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:62)
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
