Hello,
Thanks Vijay.
I downloaded the last trunk and deployed it on my Windows system (I'm
using cygwin to interact with Nutch). I took back my modified conf files
and added my custom plugins into the plugin directory. But things are
not working... When I try to lunch a crawl I get the following error:
Injector: Converting injected urls to crawl db entries.
Exception in thread "main" java.io.IOException: Failed to get the
current user's information: Login failed: bash:
/cygdrive/c/cygwin/bin/groups: /bin/sh: bad interpreter: No such file or
directory
at
org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:466)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:805)
at org.apache.nutch.crawl.Injector.inject(Injector.java:157)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:113)
Af if it was looking for /bin/sh. Any idea what this might be?
Thank you,
David
-----Original Message-----
From: Vijay Krishnan [mailto:[EMAIL PROTECTED]
Sent: vendredi, 16. mai 2008 01:11
To: [email protected]
Subject: Re: unable to correctly fetch https pages
Yes, you can get Nutch out of the trunk or apply the following patch:
http://www.mail-archive.com/[EMAIL PROTECTED]/msg12
893.html
I had the same problem and this patch handled it.
Vijay
On Thu, May 15, 2008 at 9:16 AM, <[EMAIL PROTECTED]> wrote:
> I would just get Nutch out of the trunk - it's quite stable.
>
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
>
> ----- Original Message ----
> > From: POIRIER David <[EMAIL PROTECTED]>
> > To: [email protected]
> > Sent: Thursday, May 15, 2008 11:11:28 AM
> > Subject: unable to correctly fetch https pages
> >
> > Hello,
> >
> >
> >
> > I'm trying to fetch a web site using the https protocol. I'm using
Nutch
> > version 0.9 and I activated the protocol-httpclient plugin. The
hadoop
> > logs are set in debug mode.
> >
> >
> >
> > When checking the logs I can see that my seed url seems to be
fetch:
> >
> > "2008-05-15 12:19:04,341 INFO fetcher.Fetcher - fetching
> > https://www.aWebSite.xyz/aPage.htm";
> >
> >
> >
> > But none of the links on this page are actually found and the
process
> > finally crash with the following error:
> >
> >
> >
> > java.lang.ArrayIndexOutOfBoundsException: -1
> >
> > at
> > org.apache.lucene.index.MultiReader.isDeleted(MultiReader.java:113)
> >
> > at
> >
org.apache.nutch.indexer.DeleteDuplicates$InputFormat$DDRecordReader.nex
> > t(DeleteDuplicates.java:176)
> >
> > at
org.apache.hadoop.mapred.MapTask$1.next(MapTask.java:157)
> >
> > at
org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46)
> >
> > at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:175)
> >
> > at
> >
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
> >
> >
> >
> > I am 100% certain that the links on the seed page are not excluded
> > because of the regex rules used by the urlfilter-regex plugin. I
tried
> > using the urlfilter-prefix and the urlfilter-suffix plugin, I
wasn't
> > luckier.
> >
> >
> >
> > I found that a bug (NUTCH-593) generating the same error was fixed
by
> > Andrzej Bialecki in February, could this fix help me? What is the
> > easiest way for me to get this fix without actually using the
complete,
> > and I guess still unstable, version 1.0 of Nutch?
> >
> >
> >
> > Thanks,
> >
> >
> >
> > David
>
>
--
Vijay Krishnan
Founder, Infoaxe Inc.
http://www.cs.stanford.edu/~vijayk
http://www.infoaxe.com/hiring.html
