Sami Siren (JIRA) wrote:
> [
> http://issues.apache.org/jira/browse/NUTCH-361?page=comments#action_12432322
> ]
>
> Sami Siren commented on NUTCH-361:
> ----------------------------------
>
> I started to write (allready put some on svn trunk) some simple junit tests
> for the main tools (inject, generate, fetch). if you can extend some of those
> to demonstrate this problem then it would be easier to track down.
>
>
I run through it and here is my problem pop out [junit] Tests run:
1, Failures: 0, Errors: 1, Time elapsed: 4.294 sec
[junit] Test org.apache.nutch.crawl.TestGenerator FAILED
I run this on server. But I have problems run test from eclipse.
java.lang.ArithmeticException: / by zero
at
org.apache.nutch.crawl.PartitionUrlByHost.getPartition(PartitionUrlByHost.java:49)
at org.apache.hadoop.mapred.MapTask$2.collect(MapTask.java:152)
at
org.apache.nutch.crawl.Generator$SelectorInverseMapper.map(Generator.java:223)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:51)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:195)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:106)
Probably some configuration problems.
regards
Uros
>> generator create fetchlist randomly
>> -----------------------------------
>>
>> Key: NUTCH-361
>> URL: http://issues.apache.org/jira/browse/NUTCH-361
>> Project: Nutch
>> Issue Type: Bug
>> Components: fetcher
>> Affects Versions: 0.9.0
>> Environment: Java 1.5, FreeBSD 6.1
>> Reporter: Uros Gruber
>> Priority: Critical
>>
>> I noticed problems during generating fetchlist. I already post some info at
>> the users list. Today I check release 0.8 and I'm certain that problem is
>> only in version later than this. I've do testnig only on 0.8 and svn from
>> today.
>> The problem is that generator generate fetchlist from crawldb but everytime
>> i run there is different number of urls in fetchlist.
>> For example I put 6 test urls we have for testing and only 5 of 20 test
>> there were all urls listed in fetchlist, sometimes onyl one. Config was
>> always the same also when testing at version 0.8.
>> I try to debug what might go wrong but I only end up that in /tmp there were
>> all urls but somehow missed in crawl_generate
>> I also se some of
>> 2006-09-02 20:14:20,147 DEBUG conf.Configuration - java.io.IOException:
>> config(config)
>> at org.apache.hadoop.conf.Configuration.<init>(Configuration.java:76)
>> at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:87)
>> at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:98)
>> at org.apache.nutch.util.NutchJob.<init>(NutchJob.java:26)
>> at org.apache.nutch.crawl.Generator.generate(Generator.java:330)
>> at org.apache.nutch.crawl.Generator.run(Generator.java:405)
>> at org.apache.nutch.util.ToolBase.doMain(ToolBase.java:145)
>> at org.apache.nutch.crawl.Generator.main(Generator.java:372)
>> if I enable DEBUG loging but I doubt that this has anything to do with this.
>>
>
>
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Nutch-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
