Not that i'm aware of. You can create an issue in Jira if you like and provide a patch if possible.
> Thanks for the follow up. Very good to know. Seems like this would be > pretty easy to add, given that the robots meta tag is supported. > > Are there any plans to do so? > > Mike > > On Mon 23 Jan 2012 02:27:02 AM PST, Markus Jelsma wrote: > > There is currently no built-in support for the x-robots-tag header. > > > > On Sunday 22 January 2012 01:01:26 Michael Lissner wrote: > >> Hi, > >> > >> I'm doing some research on what technologies various crawlers support > >> for crawl exclusion. Without installing and figuring out Nutch, I can't > >> figure out if it supports the x-robots-tag HTTP header? > >> > >> Does anybody know? > >> > >> Thanks, > >> > >> Mike
