Hi, I think I've seen something like this in Tika recently. May be worth checking?
Julien -- DigitalPebble Ltd http://www.digitalpebble.com 2008/5/16 POIRIER David <[EMAIL PROTECTED]>: > Analysing the nutch sh script I found that the classes are actually read > inside the job file, in my case the nutch-0.9.job file. Though I haven't > figured out why this archive also contains the /plugin repository and > all the files found in the ./conf directory, the patch did its work and > I don't have the "MultiReader.java:113" error anymore. > > I still have one question: I have a few pdfs files that I'd like to > crawl that are images of scan documents. Is there a plugin that has > someking of OCR text extraction tool? > > Good week-end to all of you, > > David > > > > > > -----Original Message----- > From: POIRIER David [mailto:[EMAIL PROTECTED] > Sent: vendredi, 16. mai 2008 12:47 > To: [email protected] > Subject: RE: unable to correctly fetch https pages > > Vijav, > > I did a few things the right way this time, I think. First I created an > eclipse project containing all the sources. Second I downloaded the > patch we were talking about and instead of applying it using the patch > command, it wasn't recognized by cygwin, I simply did the changes myself > in the source code. Then, using the jar Ant command I rebuilt a > nutch-0.9.jar archive. > > I just want to confirm something with you: When using the bin/crawl > command, the needed archive is the jar file, nutch-0.9.jar in my case, > available in the $NUTCH_HOME directory? > > Again, thank you, > > David > > > -----Original Message----- > From: Vijay Krishnan [mailto:[EMAIL PROTECTED] > Sent: vendredi, 16. mai 2008 10:51 > To: [email protected] > Subject: Re: unable to correctly fetch https pages > > I haven't checked out the latest trunk, so no idea about this error. > > > Vijay > > On Fri, May 16, 2008 at 1:49 AM, POIRIER David > <[EMAIL PROTECTED]> wrote: > > Hello, > > > > Thanks Vijay. > > > > I downloaded the last trunk and deployed it on my Windows system (I'm > > using cygwin to interact with Nutch). I took back my modified conf > files > > and added my custom plugins into the plugin directory. But things are > > not working... When I try to lunch a crawl I get the following error: > > > > Injector: Converting injected urls to crawl db entries. > > Exception in thread "main" java.io.IOException: Failed to get the > > current user's information: Login failed: bash: > > /cygdrive/c/cygwin/bin/groups: /bin/sh: bad interpreter: No such file > or > > directory > > > > at > > org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:466) > > at > org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:805) > > at org.apache.nutch.crawl.Injector.inject(Injector.java:157) > > at org.apache.nutch.crawl.Crawl.main(Crawl.java:113) > > > > > > Af if it was looking for /bin/sh. Any idea what this might be? > > > > Thank you, > > > > David > > > > > > > > > > -----Original Message----- > > From: Vijay Krishnan [mailto:[EMAIL PROTECTED] > > Sent: vendredi, 16. mai 2008 01:11 > > To: [email protected] > > Subject: Re: unable to correctly fetch https pages > > > > Yes, you can get Nutch out of the trunk or apply the following patch: > > > > > http://www.mail-archive.com/[EMAIL PROTECTED]/msg12 > > 893.html > > > > I had the same problem and this patch handled it. > > > > > > Vijay > > > > On Thu, May 15, 2008 at 9:16 AM, <[EMAIL PROTECTED]> wrote: > >> I would just get Nutch out of the trunk - it's quite stable. > >> > >> > >> Otis > >> -- > >> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > >> > >> > >> > >> > >> ----- Original Message ---- > >> > From: POIRIER David <[EMAIL PROTECTED]> > >> > To: [email protected] > >> > Sent: Thursday, May 15, 2008 11:11:28 AM > >> > Subject: unable to correctly fetch https pages > >> > > >> > Hello, > >> > > >> > > >> > > >> > I'm trying to fetch a web site using the https protocol. I'm using > > Nutch > >> > version 0.9 and I activated the protocol-httpclient plugin. The > > hadoop > >> > logs are set in debug mode. > >> > > >> > > >> > > >> > When checking the logs I can see that my seed url seems to be > > fetch: > >> > > >> > "2008-05-15 12:19:04,341 INFO fetcher.Fetcher - fetching > >> > https://www.aWebSite.xyz/aPage.htm"; > >> > > >> > > >> > > >> > But none of the links on this page are actually found and the > > process > >> > finally crash with the following error: > >> > > >> > > >> > > >> > java.lang.ArrayIndexOutOfBoundsException: -1 > >> > > >> > at > >> > > org.apache.lucene.index.MultiReader.isDeleted(MultiReader.java:113) > >> > > >> > at > >> > > > > org.apache.nutch.indexer.DeleteDuplicates$InputFormat$DDRecordReader.nex > >> > t(DeleteDuplicates.java:176) > >> > > >> > at > > org.apache.hadoop.mapred.MapTask$1.next(MapTask.java:157) > >> > > >> > at > > org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46) > >> > > >> > at > > org.apache.hadoop.mapred.MapTask.run(MapTask.java:175) > >> > > >> > at > >> > > > > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126) > >> > > >> > > >> > > >> > I am 100% certain that the links on the seed page are not excluded > >> > because of the regex rules used by the urlfilter-regex plugin. I > > tried > >> > using the urlfilter-prefix and the urlfilter-suffix plugin, I > > wasn't > >> > luckier. > >> > > >> > > >> > > >> > I found that a bug (NUTCH-593) generating the same error was fixed > > by > >> > Andrzej Bialecki in February, could this fix help me? What is the > >> > easiest way for me to get this fix without actually using the > > complete, > >> > and I guess still unstable, version 1.0 of Nutch? > >> > > >> > > >> > > >> > Thanks, > >> > > >> > > >> > > >> > David > >> > >> > > > > > > > > -- > > Vijay Krishnan > > Founder, Infoaxe Inc. > > http://www.cs.stanford.edu/~vijayk<http://www.cs.stanford.edu/%7Evijayk> > > http://www.infoaxe.com/hiring.html > > > > > > -- > Vijay Krishnan > Founder, Infoaxe Inc. > http://www.cs.stanford.edu/~vijayk <http://www.cs.stanford.edu/%7Evijayk> > http://www.infoaxe.com/hiring.html >
