[
https://issues.apache.org/jira/browse/NUTCH-933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Nioche resolved NUTCH-933.
---------------------------------
Resolution: Not a Problem
Fix Version/s: (was: 1.9)
Marked as not a problem. PLease reopen if necessary
> Fetcher does not save a pages Last-Modified value in CrawlDatum
> ---------------------------------------------------------------
>
> Key: NUTCH-933
> URL: https://issues.apache.org/jira/browse/NUTCH-933
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 1.2
> Reporter: Joe Kemp
>
> I added the following code in the output method just after the If (content
> !=null) statement.
> String lastModified = metadata.get("Last-Modified");
> if (lastModified !=null && !lastModified.equals("")) {
> try {
> Date lastModifiedDate =
> DateUtil.parseDate(lastModified);
>
> datum.setModifiedTime(lastModifiedDate.getTime());
> } catch (DateParseException e) {
>
> }
> }
> I now get 304 for pages that haven't changed when I recrawl. Need to do
> further testing. Might also need a configuration parameter to turn off this
> behavior, allowing pages to be forced to be refreshed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
