You can use a SegmentReader object to give you references to the FetcherOutput, ParseData, and Content objects for each page in the segment. The raw page data is encapsulated within the Content object so you can parse out whatever you want from it.
However, somebody correct me if I'm wrong, but I don't think you can update individual ArrayFile entries once they've been written. So while you're looping over each ParseData entry, you can write your updated ParseData objects to a temporary ArrayFile and replace it with the old one when you're done. Andy On 7/5/05, Fredrik Andersson <[EMAIL PROTECTED]> wrote: > Hi! > > I'm new to this list, so hello to you all. > > Here's the gig - I have crawled and indexed a bunch of pages. The HTML > Parser used in nutch only parses out the title, text, metadata and > outlinks. Is there any way to extend this set of attributes > post-crawling (i.e, without rewriting HtmlParser.java)? I'd like to > iterate all the crawled pages, access their raw data, parse out some > chunk of text and save it as a detail field or similar. > > I haven't really got the full hang of the all the connections in the > API yet, so forgive a poor guy for being a newbie. > > Big thanks in advance, > Fredrik > ------------------------------------------------------- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id492&op=click _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
