[ https://issues.apache.org/jira/browse/NUTCH-1705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tien Nguyen Manh updated NUTCH-1705: ------------------------------------ Attachment: NUTCH-1705.patch > Make configuration option for HtmlParser & TikaParser to extract text or > title for noIndex page > ----------------------------------------------------------------------------------------------- > > Key: NUTCH-1705 > URL: https://issues.apache.org/jira/browse/NUTCH-1705 > Project: Nutch > Issue Type: Improvement > Reporter: Tien Nguyen Manh > Priority: Minor > Attachments: NUTCH-1705.patch > > > Currently HtmlParser and TikaParser always skip extracting text and title for > noIndex page - page which have noIndex robots metatags. > But some parse-filter may still interested in text and title such as > NUTCH-1661, where we may decide wether to follow a page by it's language. -- This message was sent by Atlassian JIRA (v6.1.5#6160)