[Nutch-general] verifying against robots.txt

[Chetan Sahasrabudhe]

Hello,
 
    I am trying to locate code that verifies the crawl process against 
robots.txt file.
Any pointers?

Regards 
Chetan