[
https://issues.apache.org/jira/browse/NUTCH-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12471991
]
nutch.newbie commented on NUTCH-443:
------------------------------------
Dogacan:
It works rather ok, But hen I changed the parse-plugins.xml a bit just to see
if I was explicitly using the parse-rss ..
<mimeType name="text/xml">
<!-- <plugin id="parse-html" /> -->
<plugin id="parse-rss" />
</mimeType>
I got stuck in dedup phase .. I tried couple of time with content type magic
being on/off
but same error.. stack trace..
2007-02-10 16:48:35,873 DEBUG mapred.MapTask - Started thread: Sort progress
reporter for task map_e7j0jb
2007-02-10 16:48:35,877 WARN mapred.LocalJobRunner - job_kyv1oj
java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.lucene.index.MultiReader.isDeleted(MultiReader.java:109)
at
org.apache.nutch.indexer.DeleteDuplicates$InputFormat$DDRecordReader.next(DeleteDuplicates.java:176)
at org.apache.hadoop.mapred.MapTask$2.next(MapTask.java:166)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:183)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:109)
I will be doing some more testing tonight with various config changes.. But in
general without any changes it works. I however haven't tested searching yet.
Please advise me if I am doing something wrong or if you want me to test
something specifically.
Cheers
> allow parsers to return multiple Parse object, this will speed up the rss
> parser
> --------------------------------------------------------------------------------
>
> Key: NUTCH-443
> URL: https://issues.apache.org/jira/browse/NUTCH-443
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Affects Versions: 0.9.0
> Reporter: Renaud Richardet
> Assigned To: Chris A. Mattmann
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: NUTCH-443-draft-v1.patch, NUTCH-443-draft-v2.patch,
> NUTCH-443-draft-v3.patch, NUTCH-443-draft-v4.patch,
> parse-map-core-draft-v1.patch, parse-map-core-untested.patch, parsers.diff
>
>
> allow Parser#parse to return a Map<String,Parse>. This way, the RSS parser
> can return multiple parse objects, that will all be indexed separately.
> Advantage: no need to fetch all feed-items separately.
> see the discussion at
> http://www.nabble.com/RSS-fecter-and-index-individul-how-can-i-realize-this-function-tf3146271.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Nutch-developers mailing list
Nutch-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nutch-developers
