Hi All, I'm wondering what the best way is to add the ability to crawl media URLs efficiently in Nutch. I don't always need to download all the cotent at the URL (e.g. a video) but would probably like to do a head request for certain media types to check the content type and the content length. How should I go about this? I'm using Nutch from SVN.
Cheers, Pablo Mayrgundter ------------------------------------------------------- This SF.Net email is sponsored by: NEC IT Guy Games. How far can you shotput a projector? How fast can you ride your desk chair down the office luge track? If you want to score the big prize, get to know the little guy. Play to win an NEC 61" plasma display: http://www.necitguy.com/?r _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
