Thank you, James. I could try it on a MAC already. It is working.
William On Wed, Mar 14, 2012 at 11:31 PM, James Kosin <[email protected]> wrote: > William, > > Works good now. > Jorn, could you please check the latest trunk with you MAC. > > James > > On 3/14/2012 10:37 AM, [email protected] wrote: > > To fix that I am trying to create the file from the resource URI. I am > not > > sure it will work in other systems, here it works. Can you confirm if it > > works for you, James? > > > > Thank you > > William > > > > On Wed, Mar 14, 2012 at 1:55 AM, James Kosin <[email protected]> > wrote: > > > >> William, > >> > >> I found this: > >> > >> > >> > http://stackoverflow.com/questions/2308188/getresourceasstream-vs-fileinputstream > >> > >> The discussion seems to suggest another issue. > >> > >> James > >> > >> On 3/14/2012 12:52 AM, [email protected] wrote: > >>> I had to do it this way otherwise it raises an exception saying that > the > >>> stream can not be marked. I think it is required by cross-validation. > >>> > >>> It is working here. Do you think it is something related to the OS? I > >> only > >>> tried it in Linux. > >>> > >>> On Wed, Mar 14, 2012 at 1:42 AM, James Kosin <[email protected]> > >> wrote: > >>>> Hi, > >>>> > >>>> Sadly, yes.... > >>>> > >>>> All the other modules get the resource like this: > >>>>> InputStream in = POSTaggerFactoryTest.class.getClassLoader() > >>>>> > >>>>> .getResourceAsStream("opennlp/tools/postag/AnnotatedSentences.txt"); > >>>> I'm not sure that getResource() is going to work. > >>>> > >>>> I end up with errors like this: > >>>>> Tests in error: > >>>>> > >>>>> > >> > testWithNameEvaluationErrorListener(opennlp.tools.namefind.TokenNameFinderCrossValidatorTest): > >> > C:\Users\James%20Kosin\Documents\NetBeansProjects\thesis\apache\opennlp\opennlp-tools\target\test-classes\opennlp\tools\namefind\AnnotatedSentences.txt > >>>>> (The system cannot find the path specified) > >>>>> > >>>>> > >> > testWithNullResources(opennlp.tools.namefind.TokenNameFinderCrossValidatorTest): > >> > C:\Users\James%20Kosin\Documents\NetBeansProjects\thesis\apache\opennlp\opennlp-tools\target\test-classes\opennlp\tools\namefind\AnnotatedSentences.txt > >>>>> (The system cannot find the path specified) > >>>> I'll give it another try on the command line with mvn to build. > >>>> > >>>> BRB, > >>>> James > >>>> > >>>> On 3/14/2012 12:11 AM, [email protected] wrote: > >>>>> Hi James, > >>>>> > >>>>> The file "AnnotatedSentences.txt" was already in SVN. It is used by > >>>> another > >>>>> unit test. > >>>>> > >>>>> Is the test failing? > >>>>> > >>>>> Thank you, > >>>>> William > >>>>> > >>>>> On Wed, Mar 14, 2012 at 1:03 AM, James Kosin <[email protected]> > >>>> wrote: > >>>>>> -------- Original Message -------- > >>>>>> Subject: Re: svn commit: r1300236 - > >>>>>> > >>>>>> > >> > /opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java > >>>>>> Date: Wed, 14 Mar 2012 00:02:18 -0400 > >>>>>> From: James Kosin <[email protected]> > >>>>>> To: [email protected] > >>>>>> > >>>>>> > >>>>>> > >>>>>> William, > >>>>>> > >>>>>> On this, don't forget to add the "AnnotatedSentences.txt" file. > >>>>>> > >>>>>> James > >>>>>> > >>>>>> On 3/13/2012 12:48 PM, [email protected] wrote: > >>>>>>> Author: colen > >>>>>>> Date: Tue Mar 13 16:48:55 2012 > >>>>>>> New Revision: 1300236 > >>>>>>> > >>>>>>> URL: http://svn.apache.org/viewvc?rev=1300236&view=rev > >>>>>>> Log: > >>>>>>> OPENNLP-466: Added JUnit that tries to reproduce the issue, but > could > >>>>>> not reproduce the error. > >>>>>>> Modified: > >>>>>>> > >> > opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java > >>>>>>> Modified: > >> > opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java > >>>>>>> URL: > >> > http://svn.apache.org/viewvc/opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java?rev=1300236&r1=1300235&r2=1300236&view=diff > >> > ============================================================================== > >>>>>>> --- > >> > opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java > >>>>>> (original) > >>>>>>> +++ > >> > opennlp/trunk/opennlp-tools/src/test/java/opennlp/tools/namefind/TokenNameFinderCrossValidatorTest.java > >>>>>> Tue Mar 13 16:48:55 2012 > >>>>>>> @@ -17,10 +17,14 @@ > >>>>>>> > >>>>>>> package opennlp.tools.namefind; > >>>>>>> > >>>>>>> -import static org.junit.Assert.assertNotNull; > >>>>>>> +import static org.junit.Assert.*; > >>>>>>> > >>>>>>> +import java.io.ByteArrayOutputStream; > >>>>>>> import java.io.FileInputStream; > >>>>>>> +import java.util.Collections; > >>>>>>> +import java.util.Map; > >>>>>>> > >>>>>>> +import opennlp.tools.cmdline.namefind.NameEvaluationErrorListener; > >>>>>>> import opennlp.tools.util.ObjectStream; > >>>>>>> import opennlp.tools.util.PlainTextByLineStream; > >>>>>>> import opennlp.tools.util.TrainingParameters; > >>>>>>> @@ -34,6 +38,9 @@ public class TokenNameFinderCrossValidat > >>>>>>> private final String TYPE = "default"; > >>>>>>> > >>>>>>> @Test > >>>>>>> + /** > >>>>>>> + * Test that reproduces jira OPENNLP-463 > >>>>>>> + */ > >>>>>>> public void testWithNullResources() throws Exception { > >>>>>>> > >>>>>>> FileInputStream sampleDataIn = new FileInputStream(getClass() > >>>>>>> @@ -49,8 +56,37 @@ public class TokenNameFinderCrossValidat > >>>>>>> TokenNameFinderCrossValidator cv = new > >>>>>> TokenNameFinderCrossValidator("en", > >>>>>>> TYPE, mlParams, null, null); > >>>>>>> > >>>>>>> - cv.evaluate(sampleStream, 1); > >>>>>>> + cv.evaluate(sampleStream, 2); > >>>>>>> > >>>>>>> assertNotNull(cv.getFMeasure()); > >>>>>>> } > >>>>>>> + > >>>>>>> + @Test > >>>>>>> + /** > >>>>>>> + * Test that tries to reproduce jira OPENNLP-466 > >>>>>>> + */ > >>>>>>> + public void testWithNameEvaluationErrorListener() throws > >> Exception { > >>>>>>> + > >>>>>>> + FileInputStream sampleDataIn = new FileInputStream(getClass() > >>>>>>> + .getClassLoader() > >>>>>>> + > >> > .getResource("opennlp/tools/namefind/AnnotatedSentences.txt").getFile()); > >>>>>>> + ObjectStream<NameSample> sampleStream = new > >> NameSampleDataStream( > >>>>>>> + new PlainTextByLineStream(sampleDataIn.getChannel(), > >>>>>> "ISO-8859-1")); > >>>>>>> + > >>>>>>> + TrainingParameters mlParams = > >>>>>> ModelUtil.createTrainingParameters(70, 1); > >>>>>>> + mlParams.put(TrainingParameters.ALGORITHM_PARAM, > >>>>>>> + ModelType.MAXENT.toString()); > >>>>>>> + > >>>>>>> + ByteArrayOutputStream out = new ByteArrayOutputStream(); > >>>>>>> + NameEvaluationErrorListener listener = new > >>>>>> NameEvaluationErrorListener(out); > >>>>>>> + > >>>>>>> + Map<String, Object> resources = Collections.emptyMap(); > >>>>>>> + TokenNameFinderCrossValidator cv = new > >>>>>> TokenNameFinderCrossValidator("en", > >>>>>>> + TYPE, mlParams, null, resources, listener); > >>>>>>> + > >>>>>>> + cv.evaluate(sampleStream, 2); > >>>>>>> + > >>>>>>> + assertTrue(out.size() > 0); > >>>>>>> + assertNotNull(cv.getFMeasure()); > >>>>>>> + } > >>>>>>> } > >>>>>>> > >>>>>>> > >> > >
