[
https://issues.apache.org/jira/browse/NUTCH-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Allison resolved NUTCH-3020.
--------------------------------
Fix Version/s: 1.20
Resolution: Fixed
> ParseSegment should check for protocol's flags for truncation
> -------------------------------------------------------------
>
> Key: NUTCH-3020
> URL: https://issues.apache.org/jira/browse/NUTCH-3020
> Project: Nutch
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Minor
> Fix For: 1.20
>
>
> As discussed on the user list, several protocols can identify when a fetch
> has been truncated. ParseSegment only checks for the number of bytes fetched
> vs the http length header (if it exists). We should modify ParseSegment to
> check for notification of truncation from the protocols.
> I noticed this specifically with okhttp, but other protocols may flag
> truncation as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
