[
https://issues.apache.org/jira/browse/CONNECTORS-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909380#comment-13909380
]
Karl Wright commented on CONNECTORS-899:
----------------------------------------
r1570840 commits some refactoring that is needed prior to working on this
ticket.
> Consider/ignore HTTP header fields when checking for document change
> --------------------------------------------------------------------
>
> Key: CONNECTORS-899
> URL: https://issues.apache.org/jira/browse/CONNECTORS-899
> Project: ManifoldCF
> Issue Type: Improvement
> Components: Web connector
> Affects Versions: ManifoldCF 1.6
> Reporter: Florian Schmedding
> Assignee: Karl Wright
> Priority: Minor
> Labels: http
> Fix For: ManifoldCF 1.6
>
>
> The web connector does already ignore certain HTTP header fields that change
> on every request when checking for document changes. However, this is
> hardcoded. Some web servers are not properly configured and return even a new
> last-modified date on each request although the document remains the same.
> This leads to lots of unncecessary re-ingestions. It would be nice to have
> the possibility to configure the header fields that should be considerd and
> ignored.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
