Hi,

I want to ignore the following urls from crawling

for eg.

http://www.example.com/stores/abcd/merch-cats-pg/abcd.*
http://www.example.com/stores/abcd/merch-cats/abcd.*
http://www.example.com/stores/abcd/merch/abd.*


I have used regex-urlfilter.txt file  and negate the following urls:


# skip URLs containing certain characters as probable queries, etc.
[EMAIL PROTECTED]
-http://([a-z0-9]*\.)*example.com/stores/.*/merch-cats-pg\.*
-http://([a-z0-9]*\.)*example.com/stores/.*/merch-cats\.*
-http://([a-z0-9]*\.)*example.com/stores/.*/merch\.*

The above filters still don't filters all the urls.

is there any way to solve this..any alternatives??

Awaiting,

Cha



-- 
View this message in context: 
http://www.nabble.com/help-needed-%3A-filters-in-regex-urlfilter.txt-tf3441531.html#a9596460
Sent from the Nutch - User mailing list archive at Nabble.com.

Reply via email to