I can understand that I am getting a "No suitable parser found" error because we don't have a parse-pdf plugin. However, it shouldn't be looking at pdfs in the first place because we are telling it to ignore pdfs with the regex-urlfilter.txt file.
I don't see how not mentioning parse-(pdf) would cause the regex-urlfilter.txt file to not work. On Tue, Dec 21, 2010 at 4:19 PM, Anurag <[email protected]> wrote: > > Yeah , may be because of this > parse-(text|html|msexcel|mspowerpoint|msword|rss|zip) > > Pdf is not included. > On Wed, Dec 22, 2010 at 2:43 AM, scohen [via Lucene] < > [email protected]<ml-node%[email protected]> > <ml-node%[email protected]<ml-node%[email protected]> > > > > wrote: > > > I forgot to mention, in nutch-site.xml we have this property: > > > > <property> > > <name>plugin.includes</name> > > > > > <value>nutch-extensionpoints|protocol-file|protocol-http|urlfilter-regex|parse-(text|html|msexcel|mspowerpoint|msword|rss|zip)|index-(anchor|basic|more)|scoring-opic|query-(basic|more|site|url)|response-(json|xml)|summary-basic|urlnormalizer-(pass|regex|basic) > > > > </value> > > </property> > > > > On Tue, Dec 21, 2010 at 3:58 PM, Steve Cohen <[hidden email]< > http://user/SendEmail.jtp?type=node&node=2128072&i=0>> > > wrote: > > > > > in the regex-urlfilter.txt we have the following: > > > > > > > > > > > > -\.(gif|GIF|jpg|JPG|png|PNG|ico|ICO|css|sit|eps|wmf|zip|ppt|mpg|xls|XLS|gz|rpm|tgz|mov|MOV|exe|jpeg|JPEG|bmp|BMP|jpe|pcx|tif|tiff|dll|DLL|a|so|o|class|bin|ttf|pfb|pfm|afm|hqx|sea|eps|ai|ram|wav|avi|mid|mov|mpg|mpeg|mp3|ogg|dat|dta|log|bz2|jar|arj|cab|rar|tar|zip|tar.gz|upp|tgz|sdd|hdr|iso|img|gpg|gbk|fac|ghg|mdic|jnilib|dmg|3gp|m4a|m4v|wma|wmv|wrl|lzh|msi|gg|kml|kmz|skb|skp|chm|mht|html/|htm/|phtml/|ghtml/|asp/|js|jsp/|shtml/|doc|PDF|pdf|swf|xml)$ > > > > > > > > > > > So we shouldn't see any mention of pdfs, right? well in the logs I am > > > seeing this: > > > > > > 2010-12-21 15:45:04,340 WARN parse.ParseUtil - No suitable parser > found > > > when trying to parse content > > > http://www.fodors.com/pdf/fodors-south-australia.pdf of type > > > application/pdf > > > 2010-12-21 15:45:04,340 WARN fetcher.Fetcher - Error parsing: > > > http://www.fodors.com/pdf/fodors-south-australia.pdf: > > > org.apache.nutch.parse.ParseException: parser not found for > > > contentType=application/pdf url= > > > http://www.fodors.com/pdf/fodors-south-australia.pdf > > > at org.apache.nutch.parse.ParseUtil.parse(ParseUtil.java:78) > > > at > > > org.apache.nutch.fetcher.Fetcher$FetcherThread.output(Fetcher.java:879) > > > at > > > org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:647 > > > > > > Does parseutil.java not use the regex-urlfilter.txt? > > > > > > Thanks, > > > Steve Cohen > > > > > > > > > ------------------------------ > > View message @ > > > http://lucene.472066.n3.nabble.com/regex-urlfilter-txt-not-working-tp2127997p2128072.html > > > > To start a new topic under Nutch - User, email > > [email protected]<ml-node%[email protected]> > <ml-node%[email protected]<ml-node%[email protected]> > > > > To unsubscribe from Nutch - User, click here< > http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=603147&code=YW51cmFnLml0LmpvbGx5QGdtYWlsLmNvbXw2MDMxNDd8LTIwOTgzNDQxOTY= > >. > > > > > > > > -- > Kumar Anurag > > > ----- > Kumar Anurag > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/regex-urlfilter-txt-not-working-tp2127997p2128100.html > Sent from the Nutch - User mailing list archive at Nabble.com. >
