cha wrote:
Hi,
I want to ignore the following urls from crawling
for eg.
http://www.example.com/stores/abcd/merch-cats-pg/abcd.*
http://www.example.com/stores/abcd/merch-cats/abcd.*
http://www.example.com/stores/abcd/merch/abd.*
I have used regex-urlfilter.txt file and negate the following urls:
# skip URLs containing certain characters as probable queries, etc.
[EMAIL PROTECTED]
-http://([a-z0-9]*\.)*example.com/stores/.*/merch-cats-pg\.*
-http://([a-z0-9]*\.)*example.com/stores/.*/merch-cats\.*
-http://([a-z0-9]*\.)*example.com/stores/.*/merch\.*
The above filters still don't filters all the urls.
is there any way to solve this..any alternatives??
Awaiting,
Cha
Did you enable urlfilter-regex plugin in your configuration?
