Lyndon Maydwell wrote:
For example, on the sites that I'm crawling, all addresses starting with www.x are simply redirects to x.
If that's really the case (you know, it doesn't always work this way for all sites) then adjust regex-urlfilter config file to remove the www. prefix - see how it's done with the session ids.
-- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __________________________________ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com
