Yes, you can get Nutch out of the trunk or apply the following patch:
http://www.mail-archive.com/[EMAIL PROTECTED]/msg12893.html
I had the same problem and this patch handled it.
Vijay
On Thu, May 15, 2008 at 9:16 AM, <[EMAIL PROTECTED]> wrote:
> I would just get Nutch out of the trunk - it's quite stable.
>
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
>
> ----- Original Message ----
> > From: POIRIER David <[EMAIL PROTECTED]>
> > To: [email protected]
> > Sent: Thursday, May 15, 2008 11:11:28 AM
> > Subject: unable to correctly fetch https pages
> >
> > Hello,
> >
> >
> >
> > I'm trying to fetch a web site using the https protocol. I'm using Nutch
> > version 0.9 and I activated the protocol-httpclient plugin. The hadoop
> > logs are set in debug mode.
> >
> >
> >
> > When checking the logs I can see that my seed url seems to be fetch:
> >
> > "2008-05-15 12:19:04,341 INFO fetcher.Fetcher - fetching
> > https://www.aWebSite.xyz/aPage.htm";
> >
> >
> >
> > But none of the links on this page are actually found and the process
> > finally crash with the following error:
> >
> >
> >
> > java.lang.ArrayIndexOutOfBoundsException: -1
> >
> > at
> > org.apache.lucene.index.MultiReader.isDeleted(MultiReader.java:113)
> >
> > at
> > org.apache.nutch.indexer.DeleteDuplicates$InputFormat$DDRecordReader.nex
> > t(DeleteDuplicates.java:176)
> >
> > at org.apache.hadoop.mapred.MapTask$1.next(MapTask.java:157)
> >
> > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46)
> >
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:175)
> >
> > at
> > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
> >
> >
> >
> > I am 100% certain that the links on the seed page are not excluded
> > because of the regex rules used by the urlfilter-regex plugin. I tried
> > using the urlfilter-prefix and the urlfilter-suffix plugin, I wasn't
> > luckier.
> >
> >
> >
> > I found that a bug (NUTCH-593) generating the same error was fixed by
> > Andrzej Bialecki in February, could this fix help me? What is the
> > easiest way for me to get this fix without actually using the complete,
> > and I guess still unstable, version 1.0 of Nutch?
> >
> >
> >
> > Thanks,
> >
> >
> >
> > David
>
>
--
Vijay Krishnan
Founder, Infoaxe Inc.
http://www.cs.stanford.edu/~vijayk
http://www.infoaxe.com/hiring.html
